Anthropic Maps AI Model 'Thought' Processes (fortune.com)
(Friday March 28, 2025 @12:05PM (msmash)
from the closer-look dept.)
- Reference: 0176844653
- News link: https://slashdot.org/story/25/03/28/0614200/anthropic-maps-ai-model-thought-processes
- Source link: https://fortune.com/2025/03/27/anthropic-ai-breakthrough-claude-llm-black-box/
Anthropic researchers have developed a breakthrough "cross-layer transcoder" (CLT) that functions like an fMRI for large language models, [1]mapping how they process information internally . Testing on Claude 3.5 Haiku, researchers discovered the model performs longer-range planning for specific tasks -- such as selecting rhyming words before constructing poem sentences -- and processes multilingual concepts in a shared neural space before converting outputs to specific languages.
The team also confirmed that LLMs can fabricate reasoning chains, either to please users with incorrect hints or to justify answers they derived instantly. The CLT identifies interpretable feature sets rather than individual neurons, allowing researchers to trace entire reasoning processes through network layers.
[1] https://fortune.com/2025/03/27/anthropic-ai-breakthrough-claude-llm-black-box/
The team also confirmed that LLMs can fabricate reasoning chains, either to please users with incorrect hints or to justify answers they derived instantly. The CLT identifies interpretable feature sets rather than individual neurons, allowing researchers to trace entire reasoning processes through network layers.
[1] https://fortune.com/2025/03/27/anthropic-ai-breakthrough-claude-llm-black-box/
Nope, Just More Hype (Score:2)
by BrendaEM ( 871664 )
No, there is no AI.
AI is no substitute for domain knowledge. (Score:4, Insightful)
If you don't have the ability to vet the thought process of the AI, you're incapable of telling whether you're being dazzled by brilliance, or baffled by bullshit. That's the fundamental problem of AI, even when the Chain of Thought is fully exposed.
Re: (Score:2)
Sure, that is true. There are however many applications where the correctness of the answer can be proven very quickly, or where the absolute correctness is a judgement call: Is this apple ready to harvest? Is this a treatable tumour? Is this a bone and is it fractured? With electronic circuit and board design, an AI can find suitable components from a large library much more quickly than an engineer and design a circuit based on a block diagram input and then the result can be verified with a schematic r
Re: (Score:2)
so just like with human output then? do we have the ability to vet the thought process of human intelligence? understanding the process does allow for some predictability, but we should be capable of distinguishing between brilliance and bullshit regardless of process or origin. we sort of do, but our mileage varies greatly. i don't see ai fundamentally different.
Re: (Score:2)
see? that's bullshit. [1]https://i.imgflip.com/2zkm0n.j... [imgflip.com]
[1] https://i.imgflip.com/2zkm0n.jpg
thought processes are overrated (Score:1)
Take [1]the smartest person on the internet [slashdot.org]
If you told him "AI claims Haitians are eating all the cats".
There would be a thought process, and evaluation of evidence.
But if you told him "Trump says Haitians are eating all the cats".
It was believed unquestioningly.
Source is more important than the "thinking" if you're a MAGA.
[1] https://slashdot.org/~iAmWaySmarterThanYou