Supreme Court supremo ponders AI-powered judges, concludes he's not out of a job yet
(2024/01/02)
- Reference: 1704230755
- News link: https://www.theregister.co.uk/2024/01/02/supreme_court_ai_roberts/
- Source link:
US Supreme Court Chief Justice John Roberts believes that artificial intelligence will play an increasingly important role in the legal process, but he expects "human judges will be around for a while."
Roberts made that observation in his 2023 Year-End Report on the Federal Judiciary
"AI obviously has great potential to dramatically increase access to key information for lawyers and non-lawyers alike," Roberts wrote. "But just as obviously it risks invading privacy interests and dehumanizing the law."
[2]
Roberts cited the potential value of AI systems to help people who cannot afford legal representation by allowing them to prepare court filings on their own. At the same time, he cautioned that AI models have made [3]headlines for their tendency to hallucinate, "which caused the lawyers using the application to submit briefs with citations to non-existent cases. (Always a bad idea.)"
[4]
[5]
As if to underscore that concern, documents unsealed last week revealed that Michael Cohen, the attorney who previously handled legal affairs for President Donald Trump, had given his own lawyer [6]fake legal citations from Google Bard. He did so in support of a motion seeking an early end to his court-ordered supervision, following his admission of 2018 campaign finance violations.
Roberts also argued that machines cannot currently match a human judge's ability to assess the sincerity of a defendant's speech. "Nuance matters: Much can turn on a shaking hand, a quivering voice, a change of inflection, a bead of sweat, a moment’s hesitation, a fleeting break in eye contact," he wrote.
[7]
And he went on to observe that, in criminal cases where AI is used to assess flight risk, recidivism or other predictive decisions, there's ongoing controversy about due process, reliability, and biases that such systems may contain.
"At least at present, studies show a persistent public perception of a 'human-AI fairness gap,' reflecting the view that human adjudications, for all of their flaws, are fairer than whatever the machine spits out," Roberts wrote.
[8]While you holidayed, Microsoft brought Copilot to mobile devices, again
[9]New York Times sues OpenAI, Microsoft over 'millions of articles' used to train ChatGPT
[10]Trust us, says EU, our AI Act will make AI trustworthy by banning the nasty ones
[11]EU running in circles trying to get AI Act out the door
That perception was challenged in a September paper by Harvard academics Victoria Angelova, Will Dobbie, and Crystal Yang through the National Bureau of Economic Research. The [12]paper , "Algorithmic Recommendations and Human Discretion," finds that when human judges override algorithmic decisions on whether to release or detain a defendant on bail, 90 percent of the humans underperform the algorithm in spotting potential recidivists.
"This finding indicates that the typical judge in our setting is less skilled at predicting misconduct than the algorithm and that we could substantially decrease misconduct rates by automating release decisions," the authors state in their paper.
At the same time, 10 percent of judges manage to outperform the algorithm when overriding its recommendations, and are better at anticipating misconduct by defendants. The common factor among these "high-skill judges" is that they're less likely to have worked previously in law enforcement and they're better at using private information not available to the algorithm.
[13]
The paper says that low-skill judges pay more attention to demographic factors like race, while high-skill judges focus more on non-demographic issues like mental health, substance abuse, and financial resources.
Human judges no doubt will be around for a while, Roberts opines. And for the underperforming majority, it may be that AI can help make them better, at least in the context of pretrial decision-making. ®
Get our [14]Tech Resources
[1] https://www.supremecourt.gov/publicinfo/year-end/2023year-endreport.pdf
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZZSVl0HegN1th4caYXY10wAAAUY&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://www.theregister.com/2023/06/22/lawyers_fake_cases/
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZZSVl0HegN1th4caYXY10wAAAUY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZZSVl0HegN1th4caYXY10wAAAUY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://www.npr.org/2023/12/30/1222273745/michael-cohen-ai-fake-legal-cases
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZZSVl0HegN1th4caYXY10wAAAUY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[8] https://www.theregister.com/2024/01/02/copilot_android_ios_apps/
[9] https://www.theregister.com/2023/12/27/the_new_york_times_files/
[10] https://www.theregister.com/2023/12/11/eu_agrees_ai_act/
[11] https://www.theregister.com/2023/12/06/eu_ai_act_negotiations/
[12] https://www.nber.org/papers/w31747
[13] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZZSVl0HegN1th4caYXY10wAAAUY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[14] https://whitepapers.theregister.com/
Roberts made that observation in his 2023 Year-End Report on the Federal Judiciary
[1]PDF
, which has not previously touched on the topic."AI obviously has great potential to dramatically increase access to key information for lawyers and non-lawyers alike," Roberts wrote. "But just as obviously it risks invading privacy interests and dehumanizing the law."
[2]
Roberts cited the potential value of AI systems to help people who cannot afford legal representation by allowing them to prepare court filings on their own. At the same time, he cautioned that AI models have made [3]headlines for their tendency to hallucinate, "which caused the lawyers using the application to submit briefs with citations to non-existent cases. (Always a bad idea.)"
[4]
[5]
As if to underscore that concern, documents unsealed last week revealed that Michael Cohen, the attorney who previously handled legal affairs for President Donald Trump, had given his own lawyer [6]fake legal citations from Google Bard. He did so in support of a motion seeking an early end to his court-ordered supervision, following his admission of 2018 campaign finance violations.
Roberts also argued that machines cannot currently match a human judge's ability to assess the sincerity of a defendant's speech. "Nuance matters: Much can turn on a shaking hand, a quivering voice, a change of inflection, a bead of sweat, a moment’s hesitation, a fleeting break in eye contact," he wrote.
[7]
And he went on to observe that, in criminal cases where AI is used to assess flight risk, recidivism or other predictive decisions, there's ongoing controversy about due process, reliability, and biases that such systems may contain.
"At least at present, studies show a persistent public perception of a 'human-AI fairness gap,' reflecting the view that human adjudications, for all of their flaws, are fairer than whatever the machine spits out," Roberts wrote.
[8]While you holidayed, Microsoft brought Copilot to mobile devices, again
[9]New York Times sues OpenAI, Microsoft over 'millions of articles' used to train ChatGPT
[10]Trust us, says EU, our AI Act will make AI trustworthy by banning the nasty ones
[11]EU running in circles trying to get AI Act out the door
That perception was challenged in a September paper by Harvard academics Victoria Angelova, Will Dobbie, and Crystal Yang through the National Bureau of Economic Research. The [12]paper , "Algorithmic Recommendations and Human Discretion," finds that when human judges override algorithmic decisions on whether to release or detain a defendant on bail, 90 percent of the humans underperform the algorithm in spotting potential recidivists.
"This finding indicates that the typical judge in our setting is less skilled at predicting misconduct than the algorithm and that we could substantially decrease misconduct rates by automating release decisions," the authors state in their paper.
At the same time, 10 percent of judges manage to outperform the algorithm when overriding its recommendations, and are better at anticipating misconduct by defendants. The common factor among these "high-skill judges" is that they're less likely to have worked previously in law enforcement and they're better at using private information not available to the algorithm.
[13]
The paper says that low-skill judges pay more attention to demographic factors like race, while high-skill judges focus more on non-demographic issues like mental health, substance abuse, and financial resources.
Human judges no doubt will be around for a while, Roberts opines. And for the underperforming majority, it may be that AI can help make them better, at least in the context of pretrial decision-making. ®
Get our [14]Tech Resources
[1] https://www.supremecourt.gov/publicinfo/year-end/2023year-endreport.pdf
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZZSVl0HegN1th4caYXY10wAAAUY&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://www.theregister.com/2023/06/22/lawyers_fake_cases/
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZZSVl0HegN1th4caYXY10wAAAUY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZZSVl0HegN1th4caYXY10wAAAUY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://www.npr.org/2023/12/30/1222273745/michael-cohen-ai-fake-legal-cases
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZZSVl0HegN1th4caYXY10wAAAUY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[8] https://www.theregister.com/2024/01/02/copilot_android_ios_apps/
[9] https://www.theregister.com/2023/12/27/the_new_york_times_files/
[10] https://www.theregister.com/2023/12/11/eu_agrees_ai_act/
[11] https://www.theregister.com/2023/12/06/eu_ai_act_negotiations/
[12] https://www.nber.org/papers/w31747
[13] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZZSVl0HegN1th4caYXY10wAAAUY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[14] https://whitepapers.theregister.com/
Absolutely f'in wow. "Nuance matters: Much can turn on a shaking hand, a quivering voice, ..."
"Nuance matters: Much can turn on a shaking hand, a quivering voice, a change of inflection, a bead of sweat, a moment’s hesitation, a fleeting break in eye contact,"
This is from the chief honcho of the US Supreme Court? No wonder the SCOTUS is now a joke. (Pardons to the few remaining intelligent members!)
No court should be looking at body language and other physical presentations in deciding the cases before them. Imagine the failed president #45 of the US who mocked individuals with physical disabilities? Would those disabilities lose a case in front of the Supreme Court?
Roberts (and I'm sure others) don't understand that many new cases will be presented through virtual media. Unless great(?) progress is made, beads of sweat and eye contact length will not be obvious.