William Perrin, former Cabinet Office civil servant and transparency campaigner, coined the phrase ‘Schrodinger’s justice’ to, in part, describe the risks involved with adopting automated decision-making and AI in the criminal justice system. The negative implication being that large parts of court proceedings and decision-making will happen in a virtual black box away from any kind of scrutiny.

While I wholeheartedly respect the sanctity of our legal system and believe it is absolutely necessary (in such an important area of government) to ensure that new technologies are transparent and unbiased, I can’t help thinking that the furore surrounding Schrodinger’s justice is just another example of sensationalism, scaremongering and the unhealthy amount of ignorance surrounding AI.  

Ignorance breeds fear and mistrust. An example of this is The Royal Society of Arts (RSA) and YouGov’s research earlier this year which found that 83% of the British public are unfamiliar with the use of automated decision-making in the criminal justice system. Despite this self-professed lack of familiarity, 60% said they oppose its use.

July 2018 saw the first public evidence-gathering hearing of the Law Society’s Technology and Law Policy Commission on ‘Algorithms in the Justice System’. Over the next six months the commission aims to examine the use of algorithms in the justice system in England and Wales and what controls, if any, are needed to protect human rights and trust in the justice system.

I had high hopes for the hearing but was disappointed to read that the main recommendation from the first session seemed to be that the Government should set up an independent public register of artificial intelligence systems to ensure that automated decision-making by the police, courts and other justice agencies is open to public scrutiny.

For me, that’s not really dealing with the issue!

There are already a few examples of Police Forces using algorithms.  Probably the most widely publicised is Durham Police’s HART algorithm pilot, which was created to assist custody officers in deciding whether or not a suspect should be released, kept in the cell, or made eligible for a local rehabilitation programme ‘Checkpoint’.

Durham's algorithm is a black box. Therefore, it isn't possible for the system to fully explain how it makes decisions. As a result HART has faced its fair share of negative publicity and accusations of discriminating against the poor.

Michael Barton, Chief Constable, of Durham Constabulary took part in the recent ‘Algorithms in the Justice System’ hearing to defend HART and reassure everyone that it is intended as a decision support tool and would never take the kind of nuanced decisions made by custody officers.

In the current AI climate, any black box automated decision-making platform that makes it difficult, or impossible, to understand exactly how a decision was reached is asking for trouble. The House of Lords Select Committee on AI has already expressed its view that ‘…it is unacceptable to deploy any AI system that could have a substantial impact on an individual’s life, unless it can generate a full and satisfactory explanation for the decisions it will take.’

Transparency has become a technical issue that developers must get right. I’ve written extensively about how developers can build transparency into AI systems and the issues stemming from machine learning and statistical methods of data analysis. In designing algorithms to learn answers, rather than have them explicitly programmed, most machine learning techniques, by their nature, are black box.

As an example, take an image classification task using the popular deep learning technique of Convolutional Neural Networks (CNN). CNN relies on a large network of weighted nodes but when it classifies an image it’s extremely difficult to understand the features extracted to make that classification. Consequently we are clueless as to what the nodes of a CNN actually represent.

‘Explainer algorithms’ that can be applied alongside other statistical techniques are one way of tackling this issue. Techniques such as Random Decision Forests (RDF) also open the door to better understanding of feature extraction.

Ultimately I believe what we really need are more technologies clearly modelling automated decision-making on human expertise — rather than black box data. AI is designed for collaboration with people, so building human expertise into the development of these platforms and enabling AI systems to provide clear explanations for the decisions they make (in a format that those same human experts understand and can confirm) is critical to the future of automated decision-making platforms in the criminal justice system and negating the risks of Schrodinger’s justice.

Technological breakthroughs will always be polarising because there are often both benefits and risks. But the ultimate success of AI and automated decision-making hinges on the uphill battle to build public trust. More explainable and transparent AI systems is the best way to win that battle.

Ben Taylor is CEO at Rainbird and advisor to the All-Party Parliamentary Group on Artificial Intelligence (APPG AI)

0 comments