Understanding the impact of algorithms

What do the regulators say?

A good place to start is to look at the guidance set out by regulators. Regulators across the world want to put in place rules and frameworks for organisations that use algorithms and have ways to check that these regulations are being followed.

At the moment, this type of regulation is nascent. A key UK regulator for the use of algorithmic systems is the Information Commissioner’s Office (ICO) who look at information rights and data protection. They have so far published draft guidance for a UK AI auditing framework, but nothing is confirmed yet.

Nonetheless, this draft guidance is a place to start to understand what regulators are hoping to see, such as:

• The use of data protection impact assessments
• Using the minimal amount of people’s data that the algorithmic system needs to work
• Considering the security of the system and the privacy of people whose data is used in the system
• Assessing the risk of bias or discrimination from the system
• Communicating about how the system is being used and how it works
• Making sure the system doesn’t infringe people’s rights

What are the risks?

Ideally you’d consider the risks of your algorithmic system before putting it into use. Before you start to use it, you could consider doing an algorithmic risk assessment.

This is to assess possible societal impacts of an algorithmic system before it is in use (with ongoing monitoring often advised). Algorithmic risk assessments look at a whole algorithmic system and how it interacts with the people it affects to:

• Identify risks or possible negative impacts
• Collaborate with potentially impacted communities to identify concerns
• Suggest ways to reduce risks and to prevent negative impacts
• Act as a record of these risks and actions, so they can be monitored over time

The AI Now institute has recommendations for public sector organisations, which could be applied to charities and social enterprises too. These assessments are now required by the Canadian Government.

One of the challenges here is that we don’t know much about how these algorithmic risk assessments work in practice - and even less so for charities. By trying out an algorithmic risk assessment and sharing your experiences, you can help others learn from them.

Annotation: The Ada Lovelace Institute / DataKind UK report looks at four types of assessment for algorithmic systems

What is its impact?

Now that you’ve looked at the potential risks and made the decision to move forward with the algorithmic system, it’s time to start using it. After it’s been running for a little while, that’s when you can start looking at its real-world impact. This is when you might consider working with researchers or social scientists to do an algorithmic impact evaluation. This looks to understand the societal impacts of an algorithmic system on the people and/or communities it affects, once it’s in use.

An example of this is Stanford’s ‘Impact evaluation of a predictive risk modeling tool for Allegheny County’s Child Welfare Office’ in which researchers looked at an algorithmic system created for use by social workers in children’s services and whether the outcomes for children were better or worse after the system was introduced.

Check for bias

One area that is often a worry is the risk of harmful impact from bias in algorithmic systems. In data science, bias refers to systematic error. For example, an algorithmic system might be much more likely to produce incorrect results for a particular group. Bias creeps into automated and human systems everywhere, and it can have far-reaching effects on society.

So how to identify bias in your algorithmic system? A bias audit is a targeted assessment to see if a system shows bias in a particular way - such as different results for people of different genders or ethnicities. For instance, the ‘Gender Shades’ bias audit looked at facial recognition systems and found that they were worst at identifying if a face was a man’s or a woman’s if the photo was of a darker skinned woman.

Whilst they’re not necessarily comprehensive, if you think there’s a risk of a particular bias in the results of your system or a system made by someone else, that impacts people your charity works with, bias audits are a way to test for it.

Conclusion

Algorithmic systems can help organisations to better support people and communities, by aiding decisions on how to direct resources, ensuring help gets to where it is most needed. But - as with any tool used to aid decision-making - these systems can also do harm if not properly evaluated and monitored.

Taking the technical steps needed to implement and evaluate an algorithmic system can be challenging - but there is help available! DataKind UK supports social change organisations who wish to explore, implement and/or evaluate an algorithmic system (or any form of data science).

Our community of pro-bono data scientists offer their data expertise to charities, social enterprises and public sector bodies - for free. If you’d like to get data science support, please get in touch on contact@datakind.org.uk.