Technology & the criminal justice system

This blog was first published in the New Law Journal on 10 June 2022

Unfairness and discrimination can be embedded in justice technology yet there is little means of scrutiny.
 

Using tech to solve problems without properly scrutinising its efficacy or considering the regulatory framework within which it has to operate can lead to expensive and embarrassing mistakes. The criminal justice system has already faced regulatory action in connection with the Metropolitan Police’s Gangs Matrix. Following concerns the Matrix included people who posed little or no risk, the London Mayor ordered a review which, according to press reports, led to about one thousand young, black men’s names being removed.

A House of Lords report published in March, suggests the criminal justice system could be forced into another humiliating policy retreat (‘Technology rules? The advent of new technologies in the justice system’, see: bit.ly/38hbOLz). The report by the Justice and Home Affairs Committee detailed a range of concerns with the technology itself, transparency about its deployment, and the oversight of its use. The committee concluded they had uncovered ‘a new Wild West, in which new technologies are developing at a pace that public awareness, government and legislation have not kept up with’.

Concerns were raised both about the way in which products were sold to law enforcement agencies and the way in which those products were evaluated by customers. The committee found vendors were not always open about how products worked when pitching to potential buyers and that a culture of evaluating technology was not embedded into law enforcement. Police forces lacked the expertise and resources to properly assess the efficacy of what they were offered before buying and deploying it.

This reluctance of police forces to invest in processes for evaluating new technology is surprising. The value of technology depends on how effectively it allows an organisation to achieve its objectives—not on the claims made by its developers. There is a real concern that money is being wasted on technology purchased on face value that is not fit for purpose.

Lack of transparency was another issue raised in the report and is one that could have a serious knock-on effect on criminal trials. They heard evidence that the underlying technology in systems operated by law enforcement was inaccessible to users, and of a failure to be transparent with stakeholders about its deployment. The complexity of systems and developers’ claims of commercial confidentiality over their technology meant forces were operating systems without adequately understanding how they worked. Because of the decentralised procurement structures in policing there is no clear picture of how widely novel technologies are being used. This lack of transparency has the potential to affect the fairness of criminal trials. If police forces are not candid about the systems used to collate, analyse and present evidence, then lawyers will not be able to effectively test the reliability of this evidence.

The committee found oversight of the use of new technology to be lacking on both a macro and micro level. Police’s internal accountability mechanisms are complex, and it was unclear who was responsible for external oversight. The government suggested in evidence to the committee that the courts could be the ultimate backstop when it came to scrutinising the use of technology in the justice system. This was somewhat surprising given their prevailing attitude to judicial review of executive action.

Finally, the committee heard evidence that individual decision makers who took data analysis from new technologies into account did not fully understand or scrutinise that analysis when making decisions, and there was a tendency to be overly reliant on it; human involvement could be limited to clicking a button confirming the algorithmic suggestion. Evidence of such approaches is particularly concerning given the emphasis that data protection legislation puts on human intervention as a safeguard when automated decision making is contemplated.

Various recommendations were made in response to the issues raised, some of them more workable than others. The idea that a single piece of new legislation could be produced that cuts across existing legislation, and is both future proof and accessible, is unrealistic. It is not clear how a single national body responsible for kitemarking AI tools would clarify lines of accountability for police forces. A mandatory register of ‘algorithms used in relevant tools’ may be overly cumbersome given the prevalence of algorithms in everyday use, or be ineffective if it does not carry a duty to explain the algorithms in the register.

In my view, the best way to rein in the ‘Wild West’ is through the introduction of training, guidance and cultural change, and the embedding of expertise within the procurement function. Data protection legislation and the Equality Act 2010 already provide frameworks through which new technologies in the justice sector should be assessed, and their significance needs to be drawn to the attention of law enforcement. It should be made clear there is an obligation to process personal data in a way which is fair, lawful and transparent; and that there are limitations on the reliance on algorithms in the context of decision making.

Moreover, law enforcement needs to be alive to the issue of being guided by big data analysis in a way that unfairly targets minorities and marginalised groups. A critical approach to emerging technologies should be taken in the procurement process and decision makers must be probing in their evaluations of what they are being sold.

Efforts to increase awareness and implement cultural change must begin now. Without such change it is increasingly likely that the courts will have to snap the criminal justice system out of its ‘digital enchantment’ through decisions in criminal trials and judicial review, with potentially expensive and embarrassing consequences for the organisations involved.

Further Information

If you have any questions regarding this blog, please contact Fred Allen in our Public Law team.

About the Author

Fred Allen is a senior associate within the Public Law Department and International Crime Group. His clients have included businesses, trade associations, religious institutions, schools, education providers, charities, and private clients including high net worth individuals, and senior political and business figures.