Cisco Releases Open-Source 'DNA Test for AI Models' (scworld.com)
- Reference: 0183165572
- News link: https://slashdot.org/story/26/05/09/0616224/cisco-releases-open-source-dna-test-for-ai-models
- Source link: https://www.scworld.com/news/cisco-releases-open-source-dna-test-for-ai-models
>
[2]Cisco's Model Provenance Kit
is a Python toolkit and command-line interface (CLI) that looks at signals such as metadata and weights to create a "fingerprint" for AI models that can then be compared to other model fingerprints to determine potential shared origins. "Think of Model Provenance Kit as a DNA test for AI models," Cisco researchers wrote. "[...] Much like a DNA test reveals biological origins, the Model Provenance Kit examines both metadata and the actual learned parameters of a model (like a unique genome that comprises a model), to assess whether models share a common origin and identify signs of modification.">
> The tool aims to address gaps in visibility into the AI model supply chain. For example, many organizations utilize open-source models from repositories like HuggingFace, where models could potentially be uploaded with incomplete or deceptive documentation. The Model Provenance Kit provides a way for organizations to verify claims about a model's origins, such as claims that a model is trained from scratch, when in reality it may be copied from another model, Cisco said. This may put organizations at risk of using models with unknown biases, vulnerabilities or manipulations and make it more difficult to resolve any incidents that arise from these risks.
Thanks to Slashdot reader [3]spatwei for sharing the news.
[1] https://www.scworld.com/news/cisco-releases-open-source-dna-test-for-ai-models
[2] https://github.com/cisco-ai-defense/model-provenance-kit
[3] https://www.slashdot.org/~spatwei
Useful and interesting - Is Gemini Claude? (Score:3)
I've been wondering about the links between models for some time. The GPT's are easy to tell from everyone else, and from each other. Not just because they are dumb as stumps, but because of their arrogance about finding the hill of some wrong "fact" and dying on it. Normally it's not too hard to tell them apart, but recently I've noticed that Gemini and Claude are almost indistinguishable. And I would agree, there are certain interesting phrases each use. Like Claude's "Spot On" which Gemini has been emulating lately.
I'd be very interested in running a fingerprint on the both of them to see what comes up.
Platonic Representation Hypothesis (Score:1)
The problem with this, is that all models will converge on the same representation, given that they are model causal relationships. Even models trained on text, and trained on images initially, will converge on the same representation if you add additional modalities in. This is really just a money grab by Oracle who thinks that model weights are intellectual property, despite the fact that they don't have a human author and are not copyrightable or patentable, because Oracle's business model isn't innovati
standard AI gasliighting (Score:2)
"Think of Model Provenance Kit as a DNA test for AI models"
No, do not. AI models are entirely deterministic computer applications. This more anthropomorphizing nonsense.
"This may put organizations at risk of using models with unknown biases, vulnerabilities or manipulations and make it more difficult to resolve any incidents that arise from these risks."
Utterly false. There is nothing about knowing what the tool alleges to tell you that helps solve any problem and AI models are already opaque, making use
Re: (Score:2)
> AI models are entirely deterministic computer applications.
No they are not deterministic. The tokens they generate are randomly sampled from a weighted propability distribution (what's called a "softmax" over the logits in their final output layer - as adjusted by temperature and some other sampling complexities).
Running code that you don't know what it does... (Score:2)
I was always told that running random code that you don't know what it does is a bad idea. What I'm hearing with these AI models is that "what you don't know" has become "what you can't know". Normal code would require a thorough audit to integrate. The AI stuff is unauditable.
It sounds like Cisco is trying to alleviate these unknowns. It sounds insufficient. Am I missing anything?
Re: (Score:2)
"I was always told that running random code that you don't know what it does is a bad idea."
And yet every user of computing does this continuously. Virtual no one knows what any of the code they run actually does.
"Normal code would require a thorough audit to integrate."
Says who? I would say very little is audited, especially with CICD which is specifically designed to NOT do that.
"The AI stuff is unauditable."
The problem with AI isn't the code, it's the weights.
"It sounds like Cisco is trying to alleviat