News: 1767698829

  ARM Give a man a fire and he's warm for a day, but set fire to him and he's warm for the rest of his life (Terry Pratchett, Jingo)

Researchers poison stolen data to make AI systems return wrong results

(2026/01/06)


Researchers affiliated with universities in China and Singapore have devised a technique to make stolen knowledge graph data useless if incorporated into a GraphRAG AI system without consent.

Large language models (LLMs) base their predictions on training data and cannot respond effectively to queries about other data. The AI industry has dealt with that limitation through a process called retrieval-augmented generation (RAG), which gives LLMs access to external datasets. Google's AI Overviews in Search, for example, [1]use RAG to provide the underlying Gemini model with current, though not necessarily accurate, web data.

[2]GraphRAG represents Microsoft's effort to make RAG more effective. By creating semantically related data clusters called knowledge graphs (KGs), GraphRAG outperforms basic RAG when linked to an LLM-based system. The structuring of the data makes it easier for the LLM to make accurate predictions when prompted.

[3]

[4]Amazon , [5]Google , and [6]Microsoft all support GraphRAG in their respective cloud services.

[7]

[8]

In a preprint [9]paper titled Making Theft Useless: Adulteration-Based Protection of Proprietary Knowledge Graphs in GraphRAG Systems, authors Weijie Wang, Peizhuo Lv, et al. observe that enterprise KGs can cost a considerable amount to build, citing a figure of [10]$5.71 per factual statement [PDF] in the KG encompassing 21 million assertions available in [11]Cyc .

Given the potential expense, companies have an incentive to prevent KG assets from being stolen and used to build a competitive AI-oriented product – [12]a concern exhibited by publishers, authors, and other creators of media content. Companies like [13]Pfizer and [14]Siemens have invested in KGs to facilitate drug discovery and assist with manufacturing.

[15]

Academics Wang, Lv, and their co-authors propose a KG defense called AURA, which stands for "Active Utility Reduction via Adulteration." The ten authors are affiliated with the Chinese Academy of Sciences, National University of Singapore, Nanyang Technological University, and Beijing University of Technology.

[16]ChatGPT is playing doctor for a lot of US residents, and OpenAI smells money

[17]AWS raises GPU prices 15% on a Saturday, hopes you weren't paying attention

[18]Claude devs complain about surprise usage limits, Anthropic blames expiring bonus

[19]AI crawlers and fetchers are blowing up websites, with Meta and OpenAI the worst offenders

AURA, they explain in their paper, is "a novel framework that makes a stolen KG unusable to an adversary while maintaining minimal performance overhead for the GraphRAG system."

Essentially, it's a mechanism for subtly poisoning or adulterating the data that goes into a KG such that accurate retrieval requires a secret key. Unlike traditional encryption, the goal is not to deny access to cleartext; rather it's to degrade KG responses to LLMs such that predictions made without the key produce reduced accuracy and hallucinations.

Alternative approaches like watermarking may have some utility for making data theft traceable, but they don't address misuse of stolen data in a private setting. And the authors argue that encryption isn't practical.

"Fully encrypting the text and embeddings would require decrypting large portions of the graph for every query," they claim. "This process introduces prohibitive computational overhead and latency, making it unsuitable for real-world use."

[20]

The threat model here assumes that the attacker has been able to steal a KG outright but hasn't obtained the secret key. Trade secret lawsuits confirm that companies like [21]Waymo aren't keen to see their IP assets spirited away.

The researchers tested their technique by creating adulterated KGs using datasets MetaQA, WebQSP, FB15K-237, and HotpotQA, then attempted to deploy GraphRAG systems using these poisoned KGs in conjunction with various LLMs (GPT-4o, Gemini-2.5-flash, Llama-2-7b, and Qwen-2.5-7b).

The results indicate that AURA is highly effective. The models retrieved adulterated content 100 percent of the time and emitted incorrect responses to users based on that misinformation 94 percent of the time.

The technique is not perfect, the academics note, because in some cases the KG may contain both correct and incorrect (adulterated) data about a subject and the LLM may choose the correct answer.

There are techniques for detoxifying poisoned data but the authors claim their approach mostly resists checks based on semantic consistency (e.g. [22]Node2Vec ), on graph-based anomaly detection (e.g. [23]ODDBALL [PDF]), and on hybrid approaches (e.g. [24]SEKA ).

"By degrading the stolen KG's utility, AURA offers a practical solution for protecting intellectual property in GraphRAG," the authors conclude. ®

Get our [25]Tech Resources



[1] https://www.rmit.edu.au/news/media-releases-and-expert-comments/2025/jun/google-ai-overview

[2] https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/

[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2aV1AD8hTaLxIF_PVcqtEwwAAA1I&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0

[4] https://aws.amazon.com/about-aws/whats-new/2025/03/amazon-bedrock-knowledge-bases-graphrag-generally-available/

[5] https://docs.cloud.google.com/architecture/gen-ai-graphrag-spanner

[6] https://learn.microsoft.com/en-us/azure/cosmos-db/gen-ai/cosmos-ai-graph

[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aV1AD8hTaLxIF_PVcqtEwwAAA1I&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33aV1AD8hTaLxIF_PVcqtEwwAAA1I&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[9] https://arxiv.org/abs/2601.00274

[10] https://ceur-ws.org/Vol-2180/ISWC_2018_Outrageous_Ideas_paper_10.pdf

[11] https://cyc.com/

[12] https://www.theregister.com/2025/12/08/publishers_say_no_ai_scrapers/

[13] https://arxiv.org/abs/2410.04660

[14] https://www.semanticscholar.org/paper/Use-Cases-of-the-Industrial-Knowledge-Graph-at-Hubauer-Lamparter/ecc8a846aee63be0a571ece752e87d7d266bbe9a

[15] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aV1AD8hTaLxIF_PVcqtEwwAAA1I&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0

[16] https://www.theregister.com/2026/01/05/chatgpt_playing_doctor_openai/

[17] https://www.theregister.com/2026/01/05/aws_price_increase/

[18] https://www.theregister.com/2026/01/05/claude_devs_usage_limits/

[19] https://www.theregister.com/2025/08/21/ai_crawler_traffic/

[20] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33aV1AD8hTaLxIF_PVcqtEwwAAA1I&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0

[21] https://jolt.law.harvard.edu/digest/waymo-v-uber-surprise-settlement-five-days-into-trial

[22] https://arxiv.org/abs/1607.00653

[23] https://www.cs.cmu.edu/~christos/courses/826.F11/CMU-ONLY/oddball.pdf

[24] https://arxiv.org/abs/2412.04780

[25] https://whitepapers.theregister.com/



"Oh no, my LLM can't use this treasure trove of stolen data!"

SVD_NL

So, this method basically adds a bunch of junk data to real data and makes the LLM more likely to choose junk data when it queries without an encryption key?

I don't see how this actually protects against IP theft, unless the only IP you're trying to protect is the knowledge graph itself, not the underlying data as you should be able to extract that using other means. I'm sure there's cases where this has some real-world applicability, but i feel like most companies wouldn't be happy about the plaintext data being stolen, even if it is slightly obfusciated.

"I don't see how this actually protects against IP theft"

Jedit

Technically it doesn't. However, the thieves aren't the direct target. The junk data makes the output of the LLMs (even) less reliable. If the user base of an LLM - the buyers of the stolen goods - know that the output can't be relied on, then in theory they'll stop using it. That's where the thieves lose their money.

The threat model here assumes that...

that one in the corner

Any comparisons available between the likelihood of those assumptions being met and the cost/complexity of this method, given the admitted holes in it?

But on the bright side, assuming (!)* that these knowledge graphs are comparable to the knowledge graphs from the 1980s, nice to see that people are catching up after 60 years.

* and if that assumption is wrong, back to the classic issue of idiots redefining words simply so that they can no longer understand and learn from research that has already been done.

Dates and elapsed times

Anonymous Coward

Last time I checked, the 1980s were 40 years ago rather than 60!

Re: Dates and elapsed times

that one in the corner

Yeah, typos be we today. Or bad editing, take your pick.

I was originally going to talk about the earlier representations that were discussed in the 1960s but then changed to the 80s 'cos that period was coming up more often in relation to the specific term "knowledge graph", as opposed to structures that do much the same job but weren't referred to as such. The 80s settled on the term because "knowledge engineering" was all the rage back then.

But then I forgot to adjust the other "60" down to "40": bad editing.

But on the bright side, I *do* how to tap on "Reply"...

Oh what a great idea

Steve Davies 3

We need more people to poison these so called AI systems data. The more unreliable data they have to work with the better IMHO.

Hey virus/malware writers... Wanna get on our good sides? Pollute these AI datasets wherever you can. If you do, you can have a virtual one of these on me.

Re: Oh what a great idea

Sudosu

This is just starting to take off in the music industry as they are aggressively hoovering up everyone's music to make sloppy copies.

https://www.youtube.com/watch?v=xMYm2d9bmEA

Takes a lot of GPU but it seems to work.

Blackjack

Just add invistext with random text copied from X, that's not poison, that's nuclear waste!

Mollison's Bureaucracy Hypothesis:
If an idea can survive a bureaucratic review and be implemented
it wasn't worth doing.