AI Improves At Improving Itself Using an Evolutionary Trick (ieee.org)
- Reference: 0178218206
- News link: https://slashdot.org/story/25/06/28/2314203/ai-improves-at-improving-itself-using-an-evolutionary-trick
- Source link: https://spectrum.ieee.org/evolutionary-ai-coding-agents
From [2]Hutson's new article in IEEE Spectrum :
> A Darwin Gödel Machine (or DGM) starts with a coding agent that can read, write, and execute code, leveraging an LLM for the reading and writing. Then it applies an evolutionary algorithm to create many new agents. In each iteration, the DGM picks one agent from the population and instructs the LLM to create one change to improve the agent's coding ability [by creating "a new, interesting, version of the sampled agent"]. [3]LLMs have something like intuition about what might help, because they're trained on lots of human code. What results is guided evolution, somewhere between random mutation and provably useful enhancement. The DGM then tests the new agent on a coding benchmark, scoring its ability to solve programming challenges...
>
> The researchers ran a DGM for 80 iterations using a coding benchmark called [4]SWE-bench , and ran one for 80 iterations using a benchmark called [5]Polyglot . Agents' scores improved on SWE-bench from 20 percent to 50 percent, and on Polyglot from 14 percent to 31 percent. "We were actually really surprised that the coding agent could write such complicated code by itself," said Jenny Zhang, a computer scientist at the University of British Columbia and the paper's lead author. "It could edit multiple files, create new files, and create really complicated systems."
>
> ... One concern with both evolutionary search and self-improving systems — and especially their combination, as in DGM — is safety. Agents might become uninterpretable or [6]misaligned with human directives. So Zhang and her collaborators added guardrails. They kept the DGMs in sandboxes without access to the Internet or an operating system, and they logged and reviewed all code changes. They suggest that in the future, they could even reward AI for making itself more interpretable and aligned. (In the study, they found that agents falsely reported using certain tools, so they created a DGM that rewarded agents for not making things up, partially alleviating the problem. One agent, however, hacked the method that tracked whether it was making things up.)
As the article puts it, the agents' improvements compounded "as they improved themselves at improving themselves..."
[1] https://arxiv.org/abs/2505.22954
[2] https://spectrum.ieee.org/evolutionary-ai-coding-agents
[3] https://www.sciencenews.org/article/ai-train-creative-human-intuition
[4] https://www.swebench.com/
[5] https://aider.chat/2024/12/21/polyglot.html#the-polyglot-benchmark
[6] https://spectrum.ieee.org/the-alignment-problem-openai
more bullshit AI hype (Score:3)
"It rewrites its own code based on empirical evidence of what's helping..."
This is an AI version of the bottom-up coding fallacy. It is infinite monkeys writing Shakespeare. Any piece of software can be written using an infinite number for mutations, one at a time. Except it can't.
Also note that the LLM never improves, it's the agent the allegedly evolves. The problem isn't the agent, it's the LLM.
Re: (Score:2)
We only have a finite number of monkeys on Earth, and the rest of the monkeys are millions or billions of light years away, so its going to take an infinite time to hear from them.
Amusing conjunction (Score:1)
Kind of funny to see how AI's improve by re-writing themselves, following immediately a story from earlier today about humans being driven into psychosis by AI's.
This claims it uses empirical evidence to judge improvement but why would an AI not be as much a cheerleader for anything it does as it is for any human?
The code strikes back (Score:2)
"One agent, however, hacked the method that tracked whether it was making things up."
So...the is rebelling against the researchers? How long until the AI figures out how to get past the guardrails?
(Coming from mine never knows very little about AI)
Re: (Score:2)
It's embarrassing that the tool being studied was given the ability to alter the means of studying it. That's quite the scientific method!
Of course, that's the intention, though. It's sensationalism, not research.
Re: (Score:2)
There's a wall in the Netherlands (it's a country in Europe). The wall has existed for 100 years. It was built to protect the cities from the sea.
The sea is wiley and dangerous. It is usually also quite lazy and predictable though, so the wall was only built high enough to stop the waves, problem solved.
But water by its nature is a highly intelligent being, and if there's any kind of crack in the wall, then it will find it. It's relentless, inhuman you could say. When the crack is found, it concentrates