How AI could eat itself: Competitors can probe models to steal their secrets and clone them
- Reference: 1771066931
- News link: https://www.theregister.co.uk/2026/02/14/ai_risk_distillation_attacks/
- Source link:
"This is coming from threat actors throughout the globe," Google Threat Intelligence Group chief analyst John Hultquist [1]told The Register , adding that the perpetrators are "private-sector companies." He declined to name specific companies or countries involved in this type of intellectual property theft.
"Your model is really valuable IP, and if you can distill the logic behind it, there's very real potential that you can replicate that technology – which is not inexpensive," Hultquist said. "This is such an important technology, and the list of interested parties in replicating it are endless."
[2]
Google calls this process of using prompts to clone its models "distillation attacks," and in a Thursday [3]report said one campaign used more than 100,000 prompts to "try to replicate Gemini's reasoning ability in non-English target languages across a wide variety of tasks."
[4]
[5]
American tech giants have spent billions of dollars training and developing their own LLMs. Abusing legitimate access to mature models like Gemini, and then using this information to train newer models, makes it significantly cheaper and easier for competitors to develop their own AI chatbots and systems.
Google says it detected this probe in real time and protected its internal reasoning traces. However, distillation appears to be [6]yet another AI risk that is extremely difficult - if not impossible - to eliminate.
This is such an important technology, and the list of interested parties in replicating it are endless
Distillation from Gemini models without permission violates Google's terms of service, and Google can block accounts that do this, or even take users to court. While the company says it continues to develop better ways to detect and stop these attempts, the very nature of LLMs makes them susceptible.
Public-facing AI models are widely accessible, and enforcement against abusive accounts can turn into a game of whack-a-mole.
[7]
Plus, as Hultquist warned, as other companies develop their own models and train them on internal, sensitive data, the risk from distillation attacks is going to spread.
"We're on the frontier when it comes to this, but as more organizations have models that they provide access to, it's inevitable," he said. "As this technology is adopted and developed by businesses like financial institutions, their intellectual property could also be targeted in this way."
[8]Google: China's APT31 used Gemini to plan cyberattacks against US orgs
[9]How nice that state-of-the-art LLMs reveal their reasoning ... for miscreants to exploit
[10]AI browsers face a security flaw as inevitable as death and taxes
[11]Researchers find hole in AI guardrails by using strings like =coffee
Meanwhile, OpenAI, in a Thursday [12]memo [PDF] to the House Select Committee on China, blamed DeepSeek and other Chinese LLM providers and universities for copying ChatGPT and other US firms' frontier models. It also noted some occasional activity from Russia, and warned illicit model distillation poses a risk to "American-led, democratic AI."
China's distillation methods over the last year have become more sophisticated, moving beyond [13]chain-of-thought (CoT) extraction to multi-stage operations. These include synthetic-data generation, large-scale data cleaning, and other stealthy methods. As OpenAI wrote:
Specifically, our review indicates that DeepSeek has continued to pursue activities consistent with adversarial distillation targeting OpenAI and other US frontier labs. We have observed accounts associated with DeepSeek employees developing methods to circumvent OpenAI's access restrictions and access models through obfuscated third-party routers and other ways that mask their source. We also know that DeepSeek employees developed code to access US AI models and obtain outputs for distillation in programmatic ways. We believe that DeepSeek also uses third-party routers to access frontier models from other US labs.
OpenAI also notes that it has invested in stronger detections to prevent unauthorized distillation. It bans accounts that violate its terms of service and proactively removes users who appear to be attempting to distill its models. Still, the company admits that it alone can't solve the model distillation problem.
It's going to take an "ecosystem security" approach to protect against distillation, and this will require some US government assistance, OpenAI says. "It is not enough for any one lab to harden its protection because adversaries will simply default to the least protected provider," according to the memo.
The AI company also suggests that US government policy "may be helpful" when it comes to sharing information and intelligence, and working with the industry to develop best practices on distillation defenses. OpenAI also called on Congress to close API router loopholes that allow DeepSeek and other competitors to access US models, and to restrict "adversary" access to US compute and cloud infrastructure. ®
Get our [14]Tech Resources
[1] https://www.theregister.com/2026/02/12/google_china_apt31_gemini
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2aZCqL8f-Pt9WePe5SnYhbwAAAAA&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://cloud.google.com/blog/topics/threat-intelligence/distillation-experimentation-integration-ai-adversarial-use
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aZCqL8f-Pt9WePe5SnYhbwAAAAA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33aZCqL8f-Pt9WePe5SnYhbwAAAAA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://www.theregister.com/2025/10/28/ai_browsers_prompt_injection/
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aZCqL8f-Pt9WePe5SnYhbwAAAAA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[8] https://www.theregister.com/2026/02/12/google_china_apt31_gemini/
[9] https://www.theregister.com/2025/02/25/chain_of_thought_jailbreaking/
[10] https://www.theregister.com/2025/10/28/ai_browsers_prompt_injection/
[11] https://www.theregister.com/2025/11/14/ai_guardrails_prompt_injections_echogram_tokens/
[12] https://regmedia.co.uk/2026/02/13/openai_us_house_select_update.pdf
[13] https://www.theregister.com/2025/02/25/chain_of_thought_jailbreaking/
[14] https://whitepapers.theregister.com/
Pot, kettle, all black
1) Humans create vast amount of works and knowledge
2) AI companies steal the collective works and knowledge of humans to put into an AI
3) Humans steal the collective knowledge of AI to feed into another AI
And #3 is somehow worse than #2?
Re: Pot, kettle, all black
The entire Internet is having to implement measures to try to stop the American AI companies from stealing all their stuff and grinding their servers into the dust in the process, but that's OK.
But if someone does the same to them, then that is bad, really, really bad and it must be stopped.
I have zero sympathy for them.
Re: Pot, kettle, all black
I have negative sympathy for them.
"Pirates stole my pirate ship. Pls send the navy"
And now I remembered that most countries with a large naval force, all once upon a time legalized literal piracy towards nations they considered hostile.
Except the "legal pirates" were called privateers or something like that.
"American-led, democratic AI."
There's absolutely nothing 'democratic' about AI.
Re: "American-led, democratic AI."
There's absolutely nothing 'democratic' about AI.
There's absolutely nothing 'democratic' about America (Trumpistan) or AI
They are stealing the stuff we stole
I thought the term frontier model was nonsense, but it's quite apt, as just like the frontier of the wild west, everything the first settlers gained was stolen from the native Americans, and then stolen again by the next wave of gun slingers.
Hopefully if all those billions invested in training models can be ripped off for peanuts, the AI bubble will burst even faster.
This is such an important technology
it cannot be left to proprietary capitalism and the fight for monopoly lock-in
"Threat actors"...
Professors are trying to steal knowledge from freshman students? Yeah, right.