UK Lords push bill to tame rogue algorithms in public sector
- Reference: 1725957057
- News link: https://www.theregister.co.uk/2024/09/10/uk_lords_algorithms_bill/
- Source link:
There are currently no legal obligations on public authorities to be transparent about when and how they use algorithms to automate decision-making without human intervention or oversight, according to Lord Clement-Jones, Liberal Democrat peer and former chair of the Lords Select Committee on AI, who is proposing the bill.
"The [1]Post Office/Horizon scandal demonstrates the painful human cost when there aren't proper checks in place to challenge these automated systems," he said. The introduction of an EPoS and accounting system in the Post Office led to hundreds of staff being erroneously prosecuted for fraud and false accounting.
[2]
"Too often in the UK we legislate when the damage has already been done. We need to be proactive, not reactive, when it comes to protecting citizens and their interactions with new technologies. We need to be ahead of the game when it comes to regulating AI. We simply cannot risk another Horizon scandal," Lord Clement-Jones said in a statement.
[3]
[4]
The UK government has been hovering around a position on automated decision-making for some time, without actually landing.
In December 2021, the government's Centre for Data Ethics and Innovation (CDEI) published a [5]"roadmap" designed to create an AI assurance industry to support the introduction of automated analysis, decision making, and processes.
[6]
Elsewhere in Whitehall, [7]the Central Digital and Data Office developed an algorithmic transparency standard for government departments and public sector bodies. Working with the CDEI, the standard would be piloted by several public sector organizations and further developed based on feedback, it said.
In September 2021, the Department for Digital, Culture, Media & Sport (DCMS) launched a consultation that suggests it could water down individuals' rights to challenge decisions made about them by artificial intelligence afforded under the EU's GDPR, which the UK still uses. The [8]proposal for new UK data protection legislation highlighted the need "to provide human review" of AI decisions that "may, in future, not be practicable or proportionate," said the paper on updating data protection laws after leaving the EU.
[9]Keir Starmer says facial recognition tech is the answer to far-right riots
[10]UK CMA says public sector will be in cloud services probe after all
[11]UK inertia on LLMs and copyright is 'de facto endorsement'
[12]Bill advances to exonerate hundreds in Post Office Horizon scandal
The proposals resulted in the [13]Data Protection and Digital Information Bill , which has passed its committee and report stages in the House of Commons, but has yet to pass through the House of Lords. Since then, the UK changed government with the election of Labour in July.
Under Lord Clement-Jones' proposals, if an automated decision affects social security benefits, immigration, or similar, a citizen would have a right to access the information on why the decision was made so that they could challenge it. The proposed bill says public authorities should publish impact assessments of any automated or AI algorithms they use to make decisions, and to keep a transparency register on automated decision making. It also obliges the government to provide an independent dispute resolution service for people who want to challenge any decisions made about them.
In its pre-election manifesto, Labour said it would ensure the safe development and use of AI models by introducing binding regulations on the handful of companies developing the most powerful AI models and by banning the creation of sexually explicit deepfakes.
[14]
Law firm Clarkslegal said reform to UK GDPR – the UK's adoption of EU data protection law after Brexit – was unlikely under Labour. However, it said legislation to regulate AI was more likely.
"In their manifesto, they have pledged to create a new Regulatory Innovation Office which is intended to help update regulation and co-ordinate this across sectors," the law firm said in a [15]blog . "Further, the manifesto proposes to introduce binding regulation intended to ensure 'the safe development and use of AI models'. They have said that this will apply to the handful of companies developing the most powerful AI models. The manifesto is silent on when regulations will be enacted which suggests that this is not something to be expected in the first few months."
In a statement sent to The Register , the Department for Science, Innovation & Technology, said:
“Technology has huge potential to improve public services, but we know it's important to maintain the right safeguards including, where appropriate, human oversight and other forms of governance.
“We will respond to Lord Clement-Jones' Private Members’ Bill in due course.” ®
Get our [16]Tech Resources
[1] https://www.theregister.com/2024/07/29/post_office_horizon_inquiry/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZuAYxHDzv1Av2o599E13CwAAAAc&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZuAYxHDzv1Av2o599E13CwAAAAc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZuAYxHDzv1Av2o599E13CwAAAAc&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://www.theregister.com/2021/12/09/uk_roadmap_for_ai_assurance/
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZuAYxHDzv1Av2o599E13CwAAAAc&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://www.gov.uk/government/news/uk-government-publishes-pioneering-standard-for-algorithmic-transparency--2
[8] https://www.theregister.com/2021/09/10/right_to_contest_automated_ai_uk_consult/
[9] https://www.theregister.com/2024/08/05/keir_starmer_facial_recognition/
[10] https://www.theregister.com/2024/06/12/uk_cma_says_public_sector/
[11] https://www.theregister.com/2024/05/03/uk_lords_llm_copyright/
[12] https://www.theregister.com/2024/05/01/post_office_exoneration_bill/
[13] https://bills.parliament.uk/bills/3430
[14] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZuAYxHDzv1Av2o599E13CwAAAAc&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[15] https://clarkslegal.com/people/lucy-densham-brown/
[16] https://whitepapers.theregister.com/
Already there
Isn't that already there? Under UK GDPR person has a right to know how decision has been made about them.
Seems like the policy makers think that if law enforcement is not keen on enforcing existing law, they need to double the amount of law and that somehow will change anything.
Re: Already there
Under GDPR a UK citizen can ask what personal information is collected and stored and what it's used for. If the information is used to make a decision about, say, housing, tax, benefits, employment, etc. then they don't necessarily have automatic rights to know how or why the decision was made. I guess that, in theory, they could go to court if they are wealthy enough, but most of us can't afford legal redress.
What has the Horizon scandal got to do with AI? Horizon was a mixture of bad software engineering, bad project management and senior managers refusing to acknowledge reality to prop up their bonuses.
There's an assumption in UK law that the machines are reliable - i.e. they do what they are intended to do - unless there is evidence to the contrary. This was the basis of most of the Horizon prosecutions and I think that the fact that the Post Office just stated that the Horizon system worked as intended was enough to prevent the defence digging any deeper. It sounds crap to me but IANAL.
https://davidallengreen.com/2023/09/computer-says-guilty-an-introduction-to-the-evidential-presumption-that-computers-are-operating-correctly/
Accountability
I think the danger here is treating the algorithm itself as legal entity. Algorithms are specced, designed, implemented and maintained by people and I'd like to see one person - someone high up in the organization, not some poor dev - on the hook for its performance. I sort of understand that once AI is brought into the equation then its performance isn't neccessarily quantifiable in the same way that a process-charted deterministic algorithm* is, but that's no reason for people to wash their hands of outcomes that have real impacts on people's lives. If I can be held responsible for my mad dog's behaviour when she's out in public then I don't see why the chief exec of my local council can't be held responsible for the council's automatic assessment of benefits, grass cutting, bus routes, etc.
* https://flowingdata.com/2009/08/28/total-eclipse-of-the-heart-flowchart/
*(not deterministic, but I think that there's a calculable** number of random run throughs would get probably get it right, although probability was never my strong subject.)
**Left to the reader.
algorithmic transparency
something I have the very distinct feeling that anything based on ML/LLM would intrinsically lack.
Promised transparency in government and in civil (public) service has never exactly thrown much light on anything let alone some of the more scandalous episodes.
Glasnost and Perestroika also are not greatly favoured in the West it would seem.