ChatGPT-Using Lawyer Cited Its Fake Witnesses and Police Testimony in Court (reuters.com)
- Reference: 0185642436
- News link: https://slashdot.org/story/26/09/14/0611201/chatgpt-using-lawyer-cited-its-fake-witnesses-and-police-testimony-in-court
- Source link: https://www.reuters.com/legal/government/chatgpt-invented-fake-police-testimony-murder-appeal-new-mexico-high-court-says-2026-09-11/
> A defense lawyer appealing his client's murder conviction submitted a brief containing made-up police testimony and witnesses fabricated by OpenAI's ChatGPT, New Mexico's highest court said. The New Mexico Supreme Court on Wednesday fined the attorney, Stephen Aarons [for $5,000], and held him in contempt for failing to verify the accuracy of the court filing, which Aarons said he prepared with help from the AI program... and said they will refer him to an attorney disciplinary board for investigation.
>
> Aarons in a statement to Reuters said he had used ChatGPT to summarize the trial proceedings when he agreed to take up the defendant's appeal last year, and did not understand the degree to which AI could "hallucinate" facts. "I am remorseful but hopeful that the disciplinary board takes into account it was an honest mistake," he said.... Dozens of lawyers have been sanctioned for filing briefs where AI made up case citations or misquoted the law. Aarons' filing appears to have gone further, containing fabricated witness testimony in a criminal appeal...
>
> Aarons told the court at an August 21 hearing that he fed a computer-generated transcript and other case materials to ChatGPT, presuming it would generate "a bulletproof summary." The justices sounded incredulous that Aarons was not fully aware of how AI can make mistakes. "Counsel, do you watch the news? Do you listen to the radio? Do you read anything about what's going on in the world?" Justice C. Shannon Bacon said at the hearing. "Because the problem with lawyers relying on AI hallucinations is an above-the-fold story every single day."
"OpenAI did not immediately respond to a request for comment."
[1] https://www.reuters.com/legal/government/chatgpt-invented-fake-police-testimony-murder-appeal-new-mexico-high-court-says-2026-09-11/
Functionally illiterate is the new norm (Score:5, Funny)
Most people hit by the "AI revolution" that I know have degraded rapidly by the daily reliance on their best friend.
And it will get a lot worse. But on the positive side, the AI is ready, once I asked it something like "mmhhh ggghhh rrhhhrmmhhr?"
It immediately replied, "what would you like, a greasy burger, an energy drink or a cool tune to rave to, give me one number and I'll tell you how to get one".
We'll be getting there fairly soon.
Re: (Score:3)
Most of us here should be old enough to remember when people tried to do serious work under Microsoft Windows in the MS-DOS era, and it crashed if you looked at it funny. Another amusing "it's never going to work" technology story was the development of lithium ion batteries. Early attempts to design a working cell [1]kept going up in flames - literally. [thermofisher.com]
It was even one of the old running gags in Disney's Carousel of Progress that new technologies were inherently buggy.
"But we do have television - when it wor
[1] https://www.thermofisher.com/blog/materials/the-history-of-the-lithium-ion-battery/
Re: (Score:2)
I'm old enough to do a lot of work using MS-DOS. I was on a team that developed a CAD system for schematic capture and printed circuit board layout. For years I used MS-DOS for software development. I don't recall anyone saying "it's never going to work". Even if MS-DOS was a simplistic and crude platform. Mind you I did have two Microsoft Surface Pro's whose batteries expanded enough to push the displays off the machine threatening to go on fire.
Re: (Score:2)
Yeah there's a bit of that going on. But Microsoft didn't claim it was going to make most work redundant and suck up all investment from something actually useful.
AI has some irresponsible marketing bullshit that's got a lot of folks laid off and leaving the ones behind to work harder.
MS Windows created lots of IT support roles.
Re: (Score:2)
It does work, people are getting more stupid, enormous resources are misallocated in a terribly inefficient manner, money and power are concentrated, etc. and it is a process that is basically irreversible.
That it produces mountains of shit as an outcome is a feature, not a bug.
Just like ignorance of the law is not an excuse. (Score:5, Insightful)
Ignorance of a broken tool combined with zero review of your own court filings should be more than discipline, it should be disbarment.
- don't practice law without a brain
Re: Just like ignorance of the law is not an excus (Score:1)
Exactly.
$5000? It's like fining Apple one million dollars. Pocket change.
Immediate disbarment is the bare minimum. If he is too stupid/ignorant to check his own submission, he has no right to be a lawyer. Not to mention courts will be even more swamped by the AI slop than they already are with proper submissions. Soon even judges will have to use AI and then what will be the point of a justice system?
Contempt of Court (Score:4, Insightful)
There is no worse contempt of court possible than to throw an LLM-generated word salad over the court wall. This "lawyer" should never be able to work in justice again.
Better call Saul (Score:2)
I mean really ... I know it's absurdly easy to practice law in the US, but really ... come on.
Pathetic (Score:2)
At least pay to use a frontier model for the highest quality nonsense.
I have always been super contemptful of courts (Score:2)
I have always been super contemptful of courts. I look at them like you look at a court of a primitive people discovered in StarTrek who think they need to hold trial for one of the StarTrek redshirts they want to kill for some pointless stupid thing like âoeusing a communicatorâ.
Everyone watching knows this primitive court is stupid, wrong and extracting out the redshirt guy by teleporting him before they can kill him is the right thing to do.
I feel this way about our courts. You people are stupi
if they can't make them stop hallucinating (Score:2)
... what makes anyone think there's a way to come up with a kill switch?
Re: (Score:2)
Chances are that power button on your phone is not really a power button but a soft suggestion. Kill switches are dead.
Re:if they can't make them stop hallucinating (Score:5, Informative)
I don't know about any particular phone, but in many devices the long press power off is an old-fashioned kill switch that charges a capacitor, eventually reaching the voltage to turn off a transistor connected to a "power good" line on the power management IC. And inside the IC it's the same deal. So as long as the button works, it should be able to hard power off, even if the software is badly bugged.
Actually numerous times I've seen people complaining their phone or tablet wouldn't turn off, when actually the software was just frozen and they didn't know about long press power off. "May I see that?" "Wow... that's amazing!" "To you, maybe..."
Re: if they can't make them stop hallucinating (Score:2)
Not necessarily a capacitor; sometimes a little counter circuit, on the MCU but independent of the CPU and not controllable (or vetoable) by it. In either case, not prone to systemic failures of the higher-level logic.
People overestimate the difficulty of adding low-level contingencies to high-level systems, and underestimate engineers' eagerness to do so. But this is the sort of thing that EEs worry about all the time .
Re: (Score:2)
When I'm worried my phone doesn't turn itself off, I just use a small flyswatter.
(And by small, I mean large.)
(And by flyswatter, I mean hammer.)
Works every time though!
(But it's expensive! I really ought to try this long press thing one day.)
Re: (Score:1)
Anecdotally, I've had pretty solid results with gemini and grok not hallucinating when it comes to the law. Unlike these lawyers, I ask the LLMs to provide direct links to the legal code and/or case briefs, and they're usually pretty spot on, usually only disagreeing with one another in subtle ways that, in my non-lawyer mind, seem to be the kind of thing that could go either way. After all, the reason you end up in court to begin with is to have the court interpret exactly these kinds of disagreements that
Re: (Score:3)
A "kill switch" is generally a hard-switch that removes power, it's not a soft-switch that the device can ignore or bug on.
Re: (Score:2)
> ... what makes anyone think there's a way to come up with a kill switch?
If a child was acting stupid and rambling on and on nonsensically last century, we used to remember what to do as adults. Spank the shit out of them and tell them to knock it off.
And keep in mind I’m talking about what a teacher would do. Before the kid ever got home to a parent who would reinforce what the educator was teaching.
When society can’t even remember how to discipline a child properly, the fuck makes you think AI will even NEED a kill switch?
General intelligence and common sense i