News: 0180230769

  ARM Give a man a fire and he's warm for a day, but set fire to him and he's warm for the rest of his life (Terry Pratchett, Jingo)

Major AI Conference Flooded With Peer Reviews Written Fully By AI (nature.com)

(Friday November 28, 2025 @11:01AM (msmash) from the closer-look dept.)


An analysis of submissions to next year's International Conference on Learning Representations has found that roughly one in five peer reviews were [1]fully generated by AI , a discovery that came after researchers including Carnegie Mellon's Graham Neubig grew suspicious of feedback on their manuscripts that seemed unusually verbose and requested non-standard statistical analyses.

Neubig posted on X offering a reward for anyone who could scan the conference's submissions for AI-generated text, and Max Spero, CEO of detection tool developer Pangram Labs, responded the next day. Pangram screened all 19,490 studies and 75,800 peer reviews submitted to ICLR 2026, finding that 21% of reviews were fully AI-generated and more than half showed signs of AI use. The conference had permitted AI tools for polishing text but prohibited falsified content. Each reviewer was assigned five papers to review in two weeks on average -- a load that senior programme chair Bharath Hariharan described as "much higher than what has been done in the past."



[1] https://www.nature.com/articles/d41586-025-03506-6


Yo Dawg (Score:2, Insightful)

by Anonymous Coward

I heard you like AI, so I put an AI in your AI so you can AI while you AI.

8th grade (Score:3)

by ThurstonMoore ( 605470 )

I wonder if telling your AI to write at an 8th grade level would help fool detectors?

AI detectors remain garbage. (Score:5, Interesting)

by Rei ( 128717 )

At one point last week I pasted the first ~300 words or so of the King James Bible into an AI detector. It told me that over half of it was AI generated.

And seriously, considering some of the god-awful stuff passing peer review in "respectable" journals these days, like [1]a paper in AIP Advances [aip.org] that claims God is a scalar field becoming [2]a featured article [phys.org], or a [3]paper in Nature [nature.com] whose Figure 1 is [4]an unusually-crappy AI image [springernature.com] talking about "Runctitiononal Features", "Medical Fymblal", "1 Tol Line storee", etc... at the very least, getting a second opinion from an AI before approving a paper would be wise.

[1] https://pubs.aip.org/aip/adv/article/15/11/115319/3372193/Universal-consciousness-as-foundational-field-A

[2] https://phys.org/news/2025-11-consciousness-foundation-theory-nature-reality.html

[3] https://www.nature.com/articles/s41598-025-24662-9

[4] https://media.springernature.com/lw685/springer-static/image/art%3A10.1038%2Fs41598-025-24662-9/MediaObjects/41598_2025_24662_Fig1_HTML.png?as=webp

Re: (Score:3)

by ZiggyZiggyZig ( 5490070 )

> I pasted the first ~300 words or so of the King James Bible into an AI detector. It told me that over half of it was AI generated.

Probably generated by Abraham Intelligence!

King James ]Re:AI detectors remain garbage.] (Score:2)

by Geoffrey.landis ( 926948 )

> At one point last week I pasted the first ~300 words or so of the King James Bible into an AI detector. It told me that over half of it was AI generated.

Sure. It reads very stilted; no actual human would write text like that these days.

Re: (Score:1)

by godrik ( 1287354 )

>> At one point last week I pasted the first ~300 words or so of the King James Bible into an AI detector. It told me that over half of it was AI generated.

> Sure. It reads very stilted; no actual human would write text like that these days.

Obviously! It was written by God!

Re: (Score:1)

by Rei ( 128717 )

And clearly God (who as we know, is [1]a scalar field [phys.org]) is an AI. That's why there's so much "slop" in the Bible - factual errors, contradictions, different versions of the same text that heavily contradict each other, etc etc. It all makes so much more sense now!

[1] https://phys.org/news/2025-11-consciousness-foundation-theory-nature-reality.html

Re: (Score:2)

by ihadafivedigituid ( 8391795 )

Wow, the "infographic" (their word) is even worse than I expected from your description.

Re: (Score:2)

by Rei ( 128717 )

They clearly didn't even use a proper image generator - that's clearly the old crappy ChatGPT-builtin image generator. It's not like it's a useful figure with a few errors - the entire thing is sheer nonsense - the more you look at it, the worse it gets. And this is Figure 1 in a *paper in Nature*. Just insane.

This problem will decrease with time ( [1]here are [bsky.app] [2]two infographics [bsky.app] from Gemini 3 I made just by pasting in an entire very long thread on Bluesky and asking for infographics, with only a few minor bits

[1] https://cdn.bsky.app/img/feed_fullsize/plain/did:plc:whp3lnoglalzs5dcsxw25mhq/bafkreic5q3rw5vki3f44bzfdgjm3p6pz4q4rfot4w3kagtdijiram7leyu@jpeg

[2] https://cdn.bsky.app/img/feed_fullsize/plain/did:plc:whp3lnoglalzs5dcsxw25mhq/bafkreifgutq3pt2potw3y3lnd4bp5kob357udcy7yv3ka7somzn2qhcydq@jpeg

Not surprised about peer review (Score:5, Interesting)

by Geoffrey.landis ( 926948 )

I am not at all surprised about AI peer review.

Peer review is part of the fundamental basis verifying the integrity of the scientific enterprise, but it is done anonymously, gets you no credit, nobody knows whether you do a good job or a bad one, and is basically a time sink with little reward except a vague feeling that you did something useful. I personally do NOT use LLM models (for peer review or anything else), but I absolutely can see how it would be very tempting to do so, a tremendous time saver with no down side.

scammers do scam (Score:2)

by Mr. Dollar Ton ( 5495648 )

news at 11

You have taken yourself too seriously.