Some scientists can't stop using AI to write research papers
- Reference: 1714732092
- News link: https://www.theregister.co.uk/2024/05/03/ai_scientific_articles/
- Source link:
Two academic papers assert that analyzing word choice in the corpus of science publications reveals an increasing usage of AI for writing research papers. [1]One study , published in March by Andrew Gray of University College London in the UK, suggests one percent of all papers published in 2023 were written at least partially by AI. A [2]second paper published in April by a Stanford University team in the US claims this figure might range between 6.3 and 17.5 percent, depending on the topic.
Both papers looked for certain words that large language models (LLMs) use habitually, such as “intricate,” “pivotal,” and “meticulously." By tracking the use of those words across scientific literature, and comparing this to words that aren't particularly favored by AI, the two studies say they can detect an increasing reliance on machine learning within the scientific publishing community.
[3]
In Gray's paper, the use of control words like "red," "conclusion," and "after" changed by a few percent from 2019 to 2023. The same was true of other certain adjectives and adverbs until 2023 (termed the post-LLM year by Gray).
[4]
[5]
In that year use of the words "meticulous," "commendable," and "intricate," rose by 59, 83, and 117 percent respectively, while their prevalence in scientific literature hardly changed between 2019 and 2022. The word with the single biggest increase in prevalence post-2022 was “meticulously”, up 137 percent.
The Stanford paper found similar phenomena, demonstrating a sudden increase for the words "realm," "showcasing," "intricate," and "pivotal." The former two were used about 80 percent more often than in 2021 and 2022, while the latter two were used around 120 and almost 160 percent more frequently respectively.
[6]Beyond the hype, AI promises leg up for scientific research
[7]AI researchers have started reviewing their peers using AI assistance
[8]Boffins deem Google DeepMind's material discoveries rather shallow
[9]Turns out AI chatbots are way more persuasive than humans
The researchers also considered word usage statistics in various scientific disciplines. Computer science and electrical engineering were ahead of the pack when it came to using AI-preferred language, while mathematics, physics, and papers published by the journal Nature, only saw increases of between five and 7.5 percent.
The Stanford bods also noted that authors posting more preprints, working in more crowded fields, and writing shorter papers seem to use AI more frequently. Their paper suggests that a general lack of time and a need to write as much as possible encourages the use of LLMs, which can help increase output.
Potentially the next big controversy in the scientific community
[10]Using AI to help in the research process isn't anything new, and lots of boffins are open about utilizing AI to tweak experiments to achieve better results. However, using AI to actually write abstracts and other chunks of papers is very different, because the general expectation is that scientific articles are written by actual humans, not robots, and at least a couple of publishers consider [11]using LLMs to write papers to be scientific misconduct.
Using AI models can be very risky as they often produce inaccurate text, the very thing scientific literature is not supposed to do. AI models can even fabricate quotations and citations, an occurrence that infamously got two New York attorneys [12]in trouble for citing cases ChatGPT had dreamed up.
[13]
"Authors who are using LLM-generated text must be pressured to disclose this or to think twice about whether doing so is appropriate in the first place, as a matter of basic research integrity," University College London’s Gray opined.
The Stanford researchers also raised similar concerns, writing that use of generative AI in scientific literature could create "risks to the security and independence of scientific practice." ®
Get our [14]Tech Resources
[1] https://arxiv.org/abs/2403.16887
[2] https://arxiv.org/abs/2404.01268
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZjUKHEWwgjXto5kNobAI7AAAAIo&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZjUKHEWwgjXto5kNobAI7AAAAIo&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZjUKHEWwgjXto5kNobAI7AAAAIo&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://www.theregister.com/2023/08/02/beyond_the_hype_ai_promises/
[7] https://www.theregister.com/2024/03/19/ai_researchers_reviewing_peers/
[8] https://www.theregister.com/2024/04/11/google_deepmind_material_study/
[9] https://www.theregister.com/2024/04/03/ai_chatbots_persuasive/
[10] https://www.theregister.com/2023/08/02/beyond_the_hype_ai_promises/
[11] https://www.theregister.com/2023/01/27/top_academic_publisher_science_bans/
[12] https://www.theregister.com/2024/02/24/chatgpt_cuddy_legal_fees/
[13] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZjUKHEWwgjXto5kNobAI7AAAAIo&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[14] https://whitepapers.theregister.com/
Re: A solution maybe?
Like a lot of things, it's "what can we measure?". Well it's easy to measure number of papers published so we'll make that a target. Which also skews what papers get written and, in turn, what research gets done. I blame MBA types for wanting everything measured all the time.
Or the explosion of the use of the word "delve" since the release of ChatGPT
There's also [1]the outright explosion of the occurrence of the word "delve" in PubMed articles that coincides neatly with the release of ChatGPT
[1] https://twitter.com/JeremyNguyenPhD/status/1774021645709295840
Re: Or the explosion of the use of the word "delve" since the release of ChatGPT
... not to delve into the pivotal explosion of intricate rats meticulously showcasing commendably [1]huuuge penises!
(some journals are surely peer-reviewed by AI itself)
[1] https://arstechnica.com/science/2024/02/scientists-aghast-at-bizarre-ai-rat-with-huge-genitals-in-peer-reviewed-article/
Re: Or the explosion of the use of the word "delve" since the release of ChatGPT
(some journals are surely peer-reviewed by AI itself)
Not sure about the I in AI. And neither about the A - rubbish has been published in "peer-reviewed" journals a long time before the latest AI hype.
Thought things were becoming more readable.
I asked Google Gemini to comment.
While large language models can be helpful for generating ideas or summarizing information, using them to write entire academic research reports is generally discouraged for a few reasons:
Lack of Critical Thinking and Originality: These models rely on existing data and might not provide new insights or critical analysis expected in academic research.
Potential for Factual Errors: The accuracy of the generated text depends on the quality of the training data. There's a risk of including factual errors or biases present in the data.
Misrepresentation of Authorship: Research reports require clear ownership of ideas and arguments. Using a large language model can blur the lines between human and machine generated content.
It's important for academic research to be transparent, well-sourced, and demonstrate critical thinking. Large language models are better suited for assisting researchers, not replacing them entirely.
:)
Lazy fuckers
Having written a (very) few in my time I can say that with a clear conscience.
Re: Lazy fuckers
Welcome to the Future.
Shrug
Far too much stuff that should be openly available is buried behind paywalls. If AI pollutes that, I'm afraid I might be able to manage a shrug at best...
Icon: meh, whatever.
Re: Shrug
Ironically there is a huge corpus of low-grade ML articles paywalled at the usual suspects (touted by Science Direct, Springer et. al. at typically $30 a pop) specifically on the subjects of ML (and optimization for training ML). Here is an excellent Arxiv link that gives a good overview of the problem: https://arxiv.org/abs/2301.01984 (The Evolutionary Computation Methods No One Should Use). A lot of this stuff predates LLMs, but just wait ;)
Re: Shrug
The bigger problem is that a lot of the quality information and journalism is increasingly behind a login and/or paywall to prevent LLMs from scraping it because it's usually not cheap to gather quality information. That leaves a lot of nonsense and poor quality information that is easily scrapable. That deluge of crap is increasingly being written by LLMs and hoovered up again by LLMs creating some kind of information death spiral.
Meticulous Applications
having just sifted through 30+ job applications I can confirm there is a rise in the use of the word meticulously.
Re: Meticulous Applications
I do hope your efforts will be considered -- and indeed were -- commendably meticulous by all concerned. Calling the task "intricate" would be an understatement.
Suggestive finding
From the UCL paper (by far the most readable of the two cited), the test adjectives and adverbs 1 (particularly the latter) are for the most part predominantly terms we might expect in marketing copy, being in general somewhat self-congratulatory. The finding that such terms are on the increase in supposedly scientific literature has two possible (not mutually exclusive) indications. Firstly that the LLMs are primarily trained on commercial bullshit rather than on scientific papers, and secondarily that scientific writing is getting less objective. The first is quite expected and may actually assist in identifying AI generated texts, but the second, if actual, bodes badly for scientific progress.
1: Adjectives: commendable, innovative, meticulous, intricate, notable, versatile, noteworthy, invaluable, pivotal, potent, fresh, ingenious; Adverbs:, meticulously, reportedly, lucidly, innovatively, aptly, methodically, excellently, compellingly, impressively, undoubtedly, scholarly, strategically
.
A solution maybe?
" Their paper suggests that a general lack of time and a need to write as much as possible encourages the use of LLMs, which can help increase output. "
There you have it -- production line 'science', where publication productivity is key to keeping your job. It's been going on for several decades, reducing the proportion of published output that represents real scientific progress. The use of LLMs is just the next logical stage in the process, and might actually assist in addressing it by generating clearly identifiable nonsense that can be filtered out as these papers suggest.