OpenAI pulls AI text detector due to it being a bit crap
- Reference: 1690329908
- News link: https://www.theregister.co.uk/2023/07/26/openai_ai_classifier/
- Source link:
"As of July 20, 2023, the AI classifier is no longer available due to its low rate of accuracy," the biz said in a short statement [1]added to its [2]January announcement of the online tool.
"We are working to incorporate feedback and are currently researching more effective provenance techniques for text, and have made a commitment to develop and deploy mechanisms that enable users to understand if audio or visual content is AI-generated," the machine-learning lab added.
[3]
The classifier was free to use, and netizens could copy and paste into it text to check whether the material was likely generated by a computer or a person. That would be useful for determining whether an email or blog post or essay was crafted by a human. Specifically, it was powered by a large language model that ranked how much of the content was likely generated by software, from "very likely" to "unclear" to "likely."
[4]
[5]
OpenAI warned at the time its AI classifier was "not fully reliable" and admitted it was prone to incorrectly flagging human-written text as machine-written. That's the same OpenAI that says things like its ChatGPT bot should not be relied upon, but champions its use anyway.
The classifier didn't work very well on writing that had been AI-generated and edited by humans, and it struggled with prose it hadn't seen in its training dataset. It was also overconfident in its predictions: "the classifier is sometimes extremely confident in a wrong prediction," OpenAI said.
[6]OpenAI offers error-prone AI detector amid fears of a machine-stuffed future
[7]Plagiarism-sniffing Turnitin tries to find AI writing by students – with mixed grades
[8]Professor freezes student grades after ChatGPT claimed AI wrote their papers
[9]No reliable way to detect AI-generated text, boffins sigh
OpenAI launched the AI classifier after growing fears about machine content being used by students to write essays and complete homework. At launch, OpenAI urged educators to not take the model's predictions as gospel, but to use it as a guide complementing "other methods of determining the source of a piece of text."
Trying to accurately classify AI text is proving difficult. Similar tools built by other developers and companies are also unreliable, and have led to real repercussions on students' education. An instructor at the University of Texas A&M-Commerce in the United States made headlines when he withheld some grades after ChatGPT predicted their text as being AI-generated. The university has [10]since reinstated the students' scores.
[11]
Meanwhile, AI software built by Turnitin has been rolled out at schools and universities to tackle plagiarism with "98 percent confidence," though it's not clear how accurate it truly is. A [12]study conducted by computer scientists at the University of Maryland suggested the chances of the best available classifiers accurately detecting machine-written text were not much better than a coin toss.
OpenAI is still working to solve this tricky problem. Last week, the Microsoft-bankrolled lab pledged to develop digital watermarks for AI-generated content as part of its [13]promise to the Biden-Harris administration to help make next-gen machine-learning technology safe to use.
The Register asked OpenAI for further comment and any predicted release date for a new build of the classifier. ®
Get our [14]Tech Resources
[1] https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text
[2] https://www.theregister.com/2023/01/31/openai_tool_chatgpt_detection/
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZMCaZH@MRFec0upYotXeKAAAAQM&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZMCaZH@MRFec0upYotXeKAAAAQM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZMCaZH@MRFec0upYotXeKAAAAQM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://www.theregister.com/2023/01/31/openai_tool_chatgpt_detection/
[7] https://www.theregister.com/2023/04/05/turntin_plagiarism_ai/
[8] https://www.theregister.com/2023/05/17/university_chatgpt_grades/
[9] https://www.theregister.com/2023/03/21/detecting_ai_generated_text/
[10] https://www.heraldbanner.com/news/local_news/multiple-tamu-c-students-exonerated-after-being-suspected-of-using-ai-to-cheat/article_5f2fe464-f4ee-11ed-a1c0-9720c4e42469.html
[11] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZMCaZH@MRFec0upYotXeKAAAAQM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[12] https://www.theregister.com/2023/03/21/detecting_ai_generated_text/
[13] https://www.theregister.com/2023/07/21/white_house_ai_rules/
[14] https://whitepapers.theregister.com/
AI classifier is no longer available due to its low rate of accuracy
So, you've built an AI tool to detect AI text because AI text is mostly inaccurate, and that tool's output is also mostly inaccurate.
In other words a failed AI fails to detect other failed AI.
This stuff is amazing. Definitely going to replace all our jobs.
Do OpenAI really want a working LLM detector? Or is it a bluff?
Training your neural net, to turn it into a LLM, requires an amount of feedback to direct the learning process: a few short years ago we were being told of "new" techniques like GANs[1], where the training eats up even more machine cycles by training two nets to basically fight each other. So long as you have the cycles available, GANs are cheaper and quicker than using humans to read all the output and grade it. You can use other software (e.g. to stop the LLM getting away with just spitting out nonsense words that aren't in the dictionary, not allowing long repetitions of one word) but that is going to be very limited in scope.
So, what are the chances that ChatGPT was created alongside its GAN nemesis: one generates text that is as good as humans can manage, the other tries to spot the difference. When the training is complete, you hope that the generator is fooling the detector as much as possible, preferably all the time.
OpenAI just happen to have a detector available, do they? I wonder where that came from. And they have let people feed into it lots and lots of samples of the sort of text that the public want to verify.
Now OpenAI admit that the detector can not spot the differences and have taken it away. At the same time, promising to come up with some kind of watermarking scheme, just what the administration would like to see. A watermarking scheme that, if they develop it first, would be pushed by the administration (with subtle hints from Microsoft) as something that every LLM creator should make use of. For a suitable licence fee, of course.
By the way, you know those restrictions that all the big boy LLMs have? About not allowing you to use their output to train another LLM? Do you think the adversarial detector counts as an LLM? It is large, it is a model and it is only concerned with language... So if you suddenly turn around and claim to already have software that can detect ChatGPT output, how did you manage to create it? Want to prove in court that it isn't an LLM? Hey, did you know that OpenAI (or another Microsoft subsidiary) has used a very similar technique?
[1] knowing what this stands for doesn't really help, but just in case you don't know: Generative Adversarial Network.
Correlating Commentards Causes Confusion
It will be interesting to compare reactions on this article against where each person's previous posts stood on the "ChatGPT just stores the text it saw in its training and prints it out again, word for word" versus the "no it doesn't, it uses (long winded description)" debate.
You lot are all going to post as AC now, aren't you!