Click here to learn more about Golnar Pooya.
The benefits offered by artificial intelligence (AI) in the public sector are nothing short of revolutionary. Algorithms powered by machine learning have the unique ability to identify tax-evasion patterns, sift through health data to prioritize cases, and predict the spread of infectious diseases much faster than humans. Governments perform more efficiently and keep costs down as a result, with analysts calculating that AI has the potential to double annual economic growth rates in developed countries by 2035.
At the same time, the risks presented by artificial intelligence in the public sector are just as radical. Decision-making driven by lines of code can be inaccurate and discriminatory, while its implementation is often kept secret from the public. Without oversight or transparency, AI in government runs the risk of undermining public trust and making our lives less private, less free, and less equal.
It is with these clear and present dangers in mind that New Zealand made history earlier this year as the world’s first country to set standards for government use of algorithms. Now, there is a strong case to be made that the rest of the world follow suit. Today, governments are driving blind with no control over who is in the AI driving seat. It is up to legislators, therefore, to wrest back power and ensure openness, accountability, and objectivity remain at the heart of public life.
Beating Back Bias
Bias is one of the major challenges presented by algorithmic decision making in the public sector. Bias can creep into algorithms as AI systems learn to make decisions based on training data, which can include biased human decisions or reflect historical or social inequities, even if sensitive variables such as gender, race, or sexual orientation are removed.
This is especially concerning when governments are deploying algorithms to profile or generate decisions about citizens by law enforcement, immigration, welfare and health agencies. Left unchecked and the repercussions of such biases reverberate in the real-world. A ProPublica investigation, for example, revealed implicit biases in courtroom algorithms resulted in harsher sentences for people of color.
The prevalence of data bias threatens a key principle of public life: objectivity. The absence of objectivity is the absence of equality, and an unequal system is one which undercuts public trust. This is why agencies in New Zealand are required to consider Indigenous worldviews on data collection – in New Zealand, Māori are disproportionately represented in the justice and prison system – and consult with groups affected by their equations.
Answerable to The Public
The lack of objectivity gives rise to another key principle of public life under threat: accountability. If government decisions are to be made by machines, there must be an explainable process answerable to public scrutiny. In adopting this tech, the public sector must create machines which clearly demonstrate the rationale behind their recommendations — and when that rationale is called into question, there must be a safe and open appeals process.
This is where the publication of public sector algorithms becomes so important. New Zealand is expected to enact rules in the coming months which force governmental agencies to publish how their algorithms are used as well as supplying the source code. At the same time, the government is expected to establish a process for citizens to demand information on AI projects, similar to requests made under the Freedom of Information Act. This requirement for transparent systems and accountable decision making is hoped to go some way to instilling public trust into these automated programs.
Artificial intelligence is going to serve the goals with which we’re now programming it. Its implementation will follow the standards we now have the opportunity to set. It is integral, therefore, that lawmakers ensure that these systems are answerable to the public they serve.
Honest Algorithm Deployment
It is common knowledge that algorithms are being deployed across the public sector, but the specifics of this deployment — like the instances in which algorithms are used and by what agencies — remain largely unknown. For example, many law enforcement agencies around the world are secretly employing software to track people’s faces, deploy patrols where crime appears most likely, and recommend whether to grant bail. This lack of transparency is compounded when algorithmic outcomes are challenged and the software creators claim trade secrets privileges to prevent access to evidence in criminal proceedings.
The core public principle of openness is nowhere to be found in this current status quo of confidentiality. Interestingly, this will remain the case in New Zealand. Bodies like the national police, who came under fire from privacy advocates last year for introducing facial recognition technology without announcing it, and the country’s spy agencies were not among the algorithm standard signatories.
If AI is going to be successfully integrated into the public sector, it is essential that the regulatory framework inspires public confidence. This public confidence can only be achieved when governments are completely transparent and publicly announce when the technology is being deployed.
Today, it is clear that the absence of public sector standards is giving rise to secretive and questionable AI integration, and this is no way to earn public trust in systems which are ostensibly brought in to improve the functions of society.
This leaves the public sector with an urgent responsibility to design robust strategies which ensure the government is positioned to pioneer these new technologies. Governments, public bodies, regulators, private companies, technology specialists, lawyers, academics and civil society groups are just some of the actors who have a role to play in the ongoing debate. New Zealand made the first step, it is time for other world leaders to take note.