New Research finds Government AI use needs Guidelines and Regulator to avoid risks
New Zealand is a world leader in government algorithm use – but measures are needed to guard against their dangers. This is the conclusion of a New Zealand Law Foundation-funded report from the University of Otago’s Artificial Intelligence and Law in New Zealand Project (AILNZP), which was released this week.
Report co-author Professor James Maclaurin says government agencies’ use of AI algorithms is increasingly coming under scrutiny.
“We might think that a computer programme can’t be prejudiced,” says co-author Associate Professor Ali Knott. “But if the information fed to it is based on previous human decisions, then its outputs could be tainted by historic human biases. There’s also a danger that other, innocent-looking factors – postcode for instance – can serve as proxies for things like race.”
Checking algorithmic decisions for these sorts of problems means that the decision-making needs to be transparent. “You can’t check or correct a decision if you can’t see how it was made, says Knott. “But in some overseas cases, that’s been impossible, because the companies who design the algorithms won’t reveal how they work.”
So far, New Zealand has done better with this, Maclaurin says. “Unlike some countries that use commercial AI products, we’ve tended to build our government AI tools in-house, which means we know how they work. That’s a practice we strongly recommend our government continues.”
Guarding against unintended algorithmic bias, though, involves more than being able to see how the code works. “Even with the best of intentions, problems can sneak back in if we’re not careful,” warns co-author Associate Professor Colin Gavaghan.
For that reason, the report recommends that New Zealand establishes a new, independent regulator, to oversee the use of algorithms in government.
The report also warns against “regulatory placebos” – measures that make us feel like we’re being protected without actually making us any safer.
The report recommends that predictive algorithms used by government, whether developed commercially or in-house, must:
Their accuracy should also be regularly assessed, with these assessments made publicly available.
Law Foundation Director Lynda Hagen says the project received the “lion’s share” of funding distributed under the Foundation’s Information Law and Policy Project.
“We did this because we think artificial intelligence and its impacts are not well-understood in New Zealand, so research was urgently needed. We welcome the release of this Phase 1 report which provides the first significant, independent and multi-disciplinary analysis of the use of algorithms by New Zealand government agencies. The information from this work will better inform the development of stronger policy and regulation.”
The report’s co-authors were Colin Gavaghan, Alistair Knott, James MacLaurin, John Zerilli and Joy Liddicoat, all of the University of Otago.
Law Foundation is contributing up to $432,217 towards this research project through its Information Law and Policy Project