Lawyers who cited fake cases hallucinated by ChatGPT must pay
(2023/06/23)
- Reference: 1687474175
- News link: https://www.theregister.co.uk/2023/06/22/lawyers_fake_cases/
- Source link:
Attorneys who filed court documents citing cases completely invented by OpenAI's ChatGPT have been formally slapped down by a New York judge.
Judge Kevin Castel on Thursday issued an [1]opinion and order on sanctions [PDF] that found Peter LoDuca, Steven A. Schwartz, and the law firm of Levidow, Levidow & Oberman P.C. had "abandoned their responsibilities when they submitted non-existent judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatGPT, then continued to stand by the fake opinions after judicial orders called their existence into question."
Yes, you got that right: the lawyers asked ChatGPT for examples of past cases to include in their legal filings, the bot just made up some previous proceedings, and the attorneys slotted those in to help make their argument and submitted it all as usual. That is not going to fly.
[2]
Built by OpenAI, ChatGPT is a large language model attached to a chat interface that responds to text prompts and questions. It was created through a training process that involves analyzing massive amounts of text and figuring out statistically probable patterns. It will then respond to input with likely output patterns, which often make sense because they resemble familiar training text. But ChatGPT, like other large language models, is known to hallucinate – to state things that are not true. Evidently, not everyone got the memo about that.
[3]
[4]
Is this is a good moment to bring up that Microsoft is heavily marketing OpenAI's GPT family of bots, [5]pushing them deep into its cloud and Windows empire, and [6]letting them loose on people's corporate data? The same models that imagine lawsuits and [7]obituaries , and were [8]described as "incredibly limited" by the software's creator? That ChatGPT?
Timeline
In late May, Judge Castel challenged the attorneys representing plaintiff Roberto Mata, a passenger injured on a 2019 Avianca airline flight, to explain themselves when the airline's lawyers suggested the opposing counsel [9]had cited fabulated rulings .
Not only did ChatGPT invent fake cases that never existed, such as " Varghese v. China Southern Airlines Co. Ltd., 925 F.3d 1339 (11th Cir. 2009) ," but, as Schwartz told the judge in his June 6 declaration, the chatty AI model also lied when questioned about the veracity of its citation, saying the case "does indeed exist" and insisting the case can be found on Westlaw and LexisNexis, despite assertions to the contrary by the court and the defense counsel.
[10]
Screenshot of ChatGPT insisting non-existent case exists ... Click to enlarge
The attorneys eventually apologized but the judge found their contrition unconvincing because they failed to admit their mistake when the issue was initially raised by the defense on March 15, 2023. Instead, they waited until May 25, 2023, after the court had issued an [11]Order to Show Cause , to acknowledge what happened.
"Many harms flow from the submission of fake opinions," Judge Castel wrote in his sanctions order.
"The opposing party wastes time and money in exposing the deception. The court’s time is taken from other important endeavors. The client may be deprived of arguments based on authentic judicial precedents. There is potential harm to the reputation of judges and courts whose names are falsely invoked as authors of the bogus opinions and to the reputation of a party attributed with fictional conduct. It promotes cynicism about the legal profession and the American judicial system. And a future litigant may be tempted to defy a judicial ruling by disingenuously claiming doubt about its authenticity."
A future litigant may be tempted to defy a judicial ruling by disingenuously claiming doubt about its authenticity
To punish the attorneys, the judge directed each to pay a $5,000 fine to the court, to notify their client, and to notify each real judge falsely identified as the author of the cited fake cases.
Concurrently, the judge [12]dismissed [PDF] plaintiff Roberto Mata's injury claim against Avianca because more than two years had passed between the injury and the lawsuit, a time limit set by the [13]Montreal Convention .
[14]
"The lesson here is that you can't delegate to a machine the things for which a lawyer is responsible," said Stephen Wu, shareholder in Silicon Valley Law Group and chair of the American Bar Association's [15]Artificial Intelligence and Robotics National Institute , in a phone interview with The Register .
[16]We just don't get enough time, contractor tasked with fact-checking Google Bard tells us
[17]AI promises vendors 15 minutes of fame with investors
[18]AI is going to eat itself: Experiment shows people training bots are using bots
[19]OpenAI calls for tough regulation of AI while quietly seeking less of it
Wu said that Judge Castel made it clear technology has a role in the legal profession when he wrote, "there is nothing inherently improper about using a reliable artificial intelligence tool for assistance."
But that role, Wu said, is necessarily subordinate to legal professionals since [20]Rule 11 of the Federal Rules of Civil Procedure requires attorneys to take responsibility for information submitted to the court.
"As lawyers, if we want to use AI to help us write things, we need something that has been trained on legal materials and has been tested rigorously," said Wu. "The lawyer always bears responsibility for the work product. You have to check your sources."
The judge's order includes, as an exhibit, the text of the invented Varghese case atop a watermark that says, "DO NOT CITE OR QUOTE AS LEGAL AUTHORITY."
[21]
However, future large language models trained on repeated media mentions of the fictitious case may keep the lie alive a bit longer. ®
Get our [22]Tech Resources
[1] https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.54.0_3.pdf
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZJUY4tJgVU8mKE89EfYcoAAAAMk&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZJUY4tJgVU8mKE89EfYcoAAAAMk&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZJUY4tJgVU8mKE89EfYcoAAAAMk&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://www.theregister.com/2023/05/10/microsoft_copilot_ai/
[6] https://www.theregister.com/2023/06/22/microsoft_azure_ai_data/
[7] https://www.theregister.com/2023/03/02/chatgpt_considered_harmful/
[8] https://twitter.com/sama/status/1601731295792414720?lang=en
[9] https://www.theregister.com/2023/05/31/texas_ai_law_court/
[10] https://regmedia.co.uk/2023/06/22/chatgpt_mata_avianca.jpg
[11] https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.33.0_1.pdf
[12] https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.55.0_3.pdf
[13] https://www.iata.org/en/programs/passenger/mc99/
[14] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZJUY4tJgVU8mKE89EfYcoAAAAMk&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[15] https://www.americanbar.org/groups/departments_offices/abacle/nat_institutes/artificial-intelligence-robotics/
[16] https://www.theregister.com/2023/06/21/google_bard_trainers/
[17] https://www.theregister.com/2023/06/14/ai_promises_vendors_15_minutes/
[18] https://www.theregister.com/2023/06/16/crowd_workers_bots_ai_training/
[19] https://www.theregister.com/2023/06/21/openai_government_regulation/
[20] https://www.law.cornell.edu/rules/frcp/rule_11
[21] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZJUY4tJgVU8mKE89EfYcoAAAAMk&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[22] https://whitepapers.theregister.com/
Judge Kevin Castel on Thursday issued an [1]opinion and order on sanctions [PDF] that found Peter LoDuca, Steven A. Schwartz, and the law firm of Levidow, Levidow & Oberman P.C. had "abandoned their responsibilities when they submitted non-existent judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatGPT, then continued to stand by the fake opinions after judicial orders called their existence into question."
Yes, you got that right: the lawyers asked ChatGPT for examples of past cases to include in their legal filings, the bot just made up some previous proceedings, and the attorneys slotted those in to help make their argument and submitted it all as usual. That is not going to fly.
[2]
Built by OpenAI, ChatGPT is a large language model attached to a chat interface that responds to text prompts and questions. It was created through a training process that involves analyzing massive amounts of text and figuring out statistically probable patterns. It will then respond to input with likely output patterns, which often make sense because they resemble familiar training text. But ChatGPT, like other large language models, is known to hallucinate – to state things that are not true. Evidently, not everyone got the memo about that.
[3]
[4]
Is this is a good moment to bring up that Microsoft is heavily marketing OpenAI's GPT family of bots, [5]pushing them deep into its cloud and Windows empire, and [6]letting them loose on people's corporate data? The same models that imagine lawsuits and [7]obituaries , and were [8]described as "incredibly limited" by the software's creator? That ChatGPT?
Timeline
In late May, Judge Castel challenged the attorneys representing plaintiff Roberto Mata, a passenger injured on a 2019 Avianca airline flight, to explain themselves when the airline's lawyers suggested the opposing counsel [9]had cited fabulated rulings .
Not only did ChatGPT invent fake cases that never existed, such as " Varghese v. China Southern Airlines Co. Ltd., 925 F.3d 1339 (11th Cir. 2009) ," but, as Schwartz told the judge in his June 6 declaration, the chatty AI model also lied when questioned about the veracity of its citation, saying the case "does indeed exist" and insisting the case can be found on Westlaw and LexisNexis, despite assertions to the contrary by the court and the defense counsel.
[10]
Screenshot of ChatGPT insisting non-existent case exists ... Click to enlarge
The attorneys eventually apologized but the judge found their contrition unconvincing because they failed to admit their mistake when the issue was initially raised by the defense on March 15, 2023. Instead, they waited until May 25, 2023, after the court had issued an [11]Order to Show Cause , to acknowledge what happened.
"Many harms flow from the submission of fake opinions," Judge Castel wrote in his sanctions order.
"The opposing party wastes time and money in exposing the deception. The court’s time is taken from other important endeavors. The client may be deprived of arguments based on authentic judicial precedents. There is potential harm to the reputation of judges and courts whose names are falsely invoked as authors of the bogus opinions and to the reputation of a party attributed with fictional conduct. It promotes cynicism about the legal profession and the American judicial system. And a future litigant may be tempted to defy a judicial ruling by disingenuously claiming doubt about its authenticity."
A future litigant may be tempted to defy a judicial ruling by disingenuously claiming doubt about its authenticity
To punish the attorneys, the judge directed each to pay a $5,000 fine to the court, to notify their client, and to notify each real judge falsely identified as the author of the cited fake cases.
Concurrently, the judge [12]dismissed [PDF] plaintiff Roberto Mata's injury claim against Avianca because more than two years had passed between the injury and the lawsuit, a time limit set by the [13]Montreal Convention .
[14]
"The lesson here is that you can't delegate to a machine the things for which a lawyer is responsible," said Stephen Wu, shareholder in Silicon Valley Law Group and chair of the American Bar Association's [15]Artificial Intelligence and Robotics National Institute , in a phone interview with The Register .
[16]We just don't get enough time, contractor tasked with fact-checking Google Bard tells us
[17]AI promises vendors 15 minutes of fame with investors
[18]AI is going to eat itself: Experiment shows people training bots are using bots
[19]OpenAI calls for tough regulation of AI while quietly seeking less of it
Wu said that Judge Castel made it clear technology has a role in the legal profession when he wrote, "there is nothing inherently improper about using a reliable artificial intelligence tool for assistance."
But that role, Wu said, is necessarily subordinate to legal professionals since [20]Rule 11 of the Federal Rules of Civil Procedure requires attorneys to take responsibility for information submitted to the court.
"As lawyers, if we want to use AI to help us write things, we need something that has been trained on legal materials and has been tested rigorously," said Wu. "The lawyer always bears responsibility for the work product. You have to check your sources."
The judge's order includes, as an exhibit, the text of the invented Varghese case atop a watermark that says, "DO NOT CITE OR QUOTE AS LEGAL AUTHORITY."
[21]
However, future large language models trained on repeated media mentions of the fictitious case may keep the lie alive a bit longer. ®
Get our [22]Tech Resources
[1] https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.54.0_3.pdf
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZJUY4tJgVU8mKE89EfYcoAAAAMk&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZJUY4tJgVU8mKE89EfYcoAAAAMk&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZJUY4tJgVU8mKE89EfYcoAAAAMk&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://www.theregister.com/2023/05/10/microsoft_copilot_ai/
[6] https://www.theregister.com/2023/06/22/microsoft_azure_ai_data/
[7] https://www.theregister.com/2023/03/02/chatgpt_considered_harmful/
[8] https://twitter.com/sama/status/1601731295792414720?lang=en
[9] https://www.theregister.com/2023/05/31/texas_ai_law_court/
[10] https://regmedia.co.uk/2023/06/22/chatgpt_mata_avianca.jpg
[11] https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.33.0_1.pdf
[12] https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.55.0_3.pdf
[13] https://www.iata.org/en/programs/passenger/mc99/
[14] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZJUY4tJgVU8mKE89EfYcoAAAAMk&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[15] https://www.americanbar.org/groups/departments_offices/abacle/nat_institutes/artificial-intelligence-robotics/
[16] https://www.theregister.com/2023/06/21/google_bard_trainers/
[17] https://www.theregister.com/2023/06/14/ai_promises_vendors_15_minutes/
[18] https://www.theregister.com/2023/06/16/crowd_workers_bots_ai_training/
[19] https://www.theregister.com/2023/06/21/openai_government_regulation/
[20] https://www.law.cornell.edu/rules/frcp/rule_11
[21] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZJUY4tJgVU8mKE89EfYcoAAAAMk&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[22] https://whitepapers.theregister.com/
Pathological guesser
CrackedNoggin
I appreciate the mathematical-algorithmic description. Sure to put the general public to sleep and pop the hype bubble. Not sure "guess" is that much much better than "hallucinate" as they both seem imply human like volition. However, if we insist the human allegory, "pathological liar" would probably be the most accurate - in this particular case the behavior is almost perfectly indistinguishable.
Re: Pathological guesser
Throatwarbler Mangrove
How about "Clavinating” after Cliff Clavin from Cheers ?
Re: Pathological guesser
Yet Another Anonymous coward
Remember pathological liars are sometimes what you want.
I doubt Wordsworth really Wandered Lonely As A Cloud and most of Shakespeare's reports about tragic deaths in Verona was about as accurate as Fox News
Re: Pathological guesser
jake
There are many huge differences between being a pathological liar and being a story teller.
It's not a GOLUM, either.
jake
The golum concept includes animation and thus is an incorrect simile.
Rather, today's AI is mostly a marketing exercise that doesn't work coupled to simple machine learning and huge databases that are demonstrably full of incorrect, incomplete and incompatible data, and are otherwise corrupt and stale. Garbage in, garbage out.
It CAN NOT work as advertised, not on a grand scale. Not today, and not any time in the future.
Please don't glorify it.
The lesson extrapolated
yetanotheraoc
"The lesson here is that you can't delegate to a _______ the things for which a ______ is responsible,"
Machine and lawyer is only one possible pair with which to irresponsibly fill the blanks.
I really don't like the term "hallucinate" for this behavior. The reality is that these GOLEMs are executing weighted random walks. That their output fails to match a series of phonemes that constitute a "true" sentence is not a malfunction in any way. It does not result from any error in the GOLEM's input, nor in the processing of said input. In fact, because the temperature is selectable, this undesired behavior is tunable.
What these GOLEMs are doing is best classified as guessing. The goal of these projects is to convince enough people that these guesses are useful. Mis-attributing what is happening in the first place is useful to their marketing, but should be banned or heartily mocked in the press.