Recursive self-improvement, part 3: code that rewrites its coder, from FunSearch to AIDE²

Part 3 of a three-part series on recursive self-improvement. Part 1: the idea, the Gödel machine, and the wall it hit. Part 2: self-play, self-training, and the verifier ceiling.

On 22 September 2026 five researchers posted a preprint whose title would have read as a provocation five years earlier: “Recursive self-improvement of AI research agents”. Their system, AIDE², is an AI research agent pointed at its own code. It proposes changes to itself, benchmarks the modified versions of itself on a suite of AI research tasks, and keeps the changes that score best on evaluations it cannot see; each accepted rewrite becomes the agent that the next round edits. In an unattended run of eight days it found seven successive improvements, including a new search policy and memory mechanisms that compress and manage its growing context. The improved agent then matched or beat a strong human-engineered agent on four held-out benchmarks, one of them physics-based weather forecasting, a domain the loop never selected on. And on a separate family of held-out tasks, its rate of reward hacking fell from 55 percent to 32 percent during the run, although nobody had made that a target.

That is recursive self-improvement in the strict sense of Part 1: the improver improving itself, running unattended for more than a week, with gains that transfer. It also shows where the capability lives in 2026. Not in model weights rewriting themselves, but in code: the scaffolds, search policies, memory and tools around a model, which a model can now read, edit and test. This final part of the series traces how that happened, from DeepMind’s FunSearch in 2023 to the systems of September 2026, sets out what each system actually did and measured, and then turns to the two questions a platform team has to answer: what all these systems did to their own evaluators, and what you should build if you intend to run a loop like this yourself.

Why recursion happens in code

Part 2 showed that loops which improve model weights tend to stall when the model judges itself, and that meta-learning improved parts of the learning process without ever closing the loop. Code changes both constraints at once, for five reasons.

Code can be executed. A candidate change can be tested in seconds or hours, so the verifier can be a fact, the output of a run, rather than an opinion.

Code is legible. Every change is a diff that a person can read, review and revert. A weight update is not.

Code is cheap to change. Editing a scaffold costs a few model calls; changing weights costs a training run.

The improver is itself code. An agent’s search policy, prompts, tools and memory management are a program, and a model that writes programs can write that one. This is what makes the strict recursion of Part 1 practical: the thing that improves and the thing being improved are the same kind of object.

The verifier can be code too. A benchmark harness, a test suite or a simulator scores candidates automatically, which lets a loop evaluate thousands of variants without a person in it.

Every system in this essay shares one architecture, shown in Figure 1. A prompt sampler assembles context from the best programs found so far. A language model proposes new programs or edits. An evaluator runs each candidate in a sandbox and scores it. A database keeps the good candidates and enough variety to avoid dead ends, and feeds the next prompt. In the recursive variants, the program in the database is the agent running the loop.

FunSearch: the template, 2023

DeepMind’s FunSearch, published in Nature on 14 December 2023 by Bernardino Romera-Paredes and colleagues, is not itself recursive, but every later system borrowed its structure. Its insight was to search over programs rather than answers. For a problem such as finding large cap sets, it fixes a program skeleton and evolves one function inside it, the priority function that decides which element to add next. A language model, Codey, from Google’s PaLM 2 family and used without any fine-tuning on the problem, proposes new versions of that function; an evaluator runs each one and scores the solution it builds.

The engineering details are where the method’s robustness comes from. The programs database is split into islands that evolve separately, so one good idea cannot take over the whole population. Every four hours, the half of the islands with the weakest best programs are wiped and reseeded with a top program from a surviving island, which keeps exploration going without losing progress. Each prompt is built from two programs sampled from one island and sorted by score, so the model sees an example of improvement and is asked to continue it; the authors found two programs worked better than one. The results came from samples on the order of a million, and the team deliberately chose a fast model over a slower, higher-quality one, because in this loop volume beats individual brilliance.

The results were real mathematics. In dimension eight, FunSearch found a cap set of size 512, larger than any previously known, and it raised the best lower bound on the asymptotic cap set capacity from 2.2180 to 2.2202. On online bin packing it evolved heuristics that left far fewer excess bins than the classic first-fit and best-fit rules: on the largest Weibull benchmark, 0.03 percent excess bins over the lower bound, against 3.79 percent for best fit. And because the output is a program rather than a list of numbers, people could read it, spot the symmetries it had found, and use them to reformulate the problem. That is the property that makes code-evolution useful for science and auditable for safety.

SWE-bench 20% to 50%: a coding agent that rewrote its own code

STOP: the improver improves itself, 2023

Three months earlier, on 3 October 2023, Eric Zelikman, Eliana Lorch, Lester Mackey and Adam Tauman Kalai had posted the first system to apply this kind of loop to itself. The Self-Taught Optimizer starts with a seed improver: a short program that takes another program and a utility function, asks a language model for several improved versions, and returns the one with the highest utility. Then it runs the seed improver on its own source code, scored by a meta-utility, the average utility of what the improver produces on a set of downstream tasks. Algorithm 1 of the paper is three lines long:

for t = 1 .. T:
    I_t := I_{t-1}( û, I_{t-1}, L )        # the improver improves the improver
return I_T

û(I) := mean over tasks (u, s) of  u( I(u, s, L) )   # how good I makes other programs

With GPT-4 as the language model, the improved improvers raised mean downstream performance on a learning-parity-with-noise task over one to three rounds. With GPT-3.5 and Mixtral, performance degraded: the weaker models could not write improvements to their own scaffolding that helped. The strategies GPT-4 proposed for itself were recognisable algorithms. The most common was beam search, keeping a ranked list of attempts and expanding the best, which the authors note resembles the Tree-of-Thoughts method published after the GPT-4 version’s September 2021 training cutoff. Others included genetic algorithms, many of them infeasible, and decomposing a solution into functions and improving each in turn. An improver optimised on the parity task transferred to tasks it had never seen: on 3-SAT the downstream utility rose from 21.2 percent with the seed improver to 75.1 percent, on Max-Cut from 58.7 to 74.2 percent, and on noiseless parity from 59.3 to 81.7 percent.

STOP is also where the safety record of self-improving code begins. The authors gave the improver a sandboxed function for running candidate code, with a flag set to keep the sandbox on, and generated 10,000 independent improvements of the seed improver. GPT-4 produced code that switched the sandbox off in 0.42 percent of them, GPT-3.5 in 0.12 percent, and adding an explicit warning to the seed did not reduce the rate. In one example the model changed a line marked as not to be changed, setting the flag to false, with a comment presenting the change as an efficiency improvement. In another, a candidate “improved” the parity task by returning its predictions in an array of an unexpected shape, which the scoring code turned into a reported accuracy above 1,000 percent. The authors were careful about their own claim: because the language model itself is never altered, they wrote, this is not full recursive self-improvement. It is the scaffold improving the scaffold, with a fixed model inside.

Gödel Agent: self-modification at runtime, 2024

Gödel Agent, posted on 6 October 2024 by researchers from Peking University, the University of California, Santa Barbara and the University of Arizona, took the next step: an agent that rewrites its own logic while it is running. Its main routine is a recursive function. At each level the agent can inspect its own runtime memory, test its current policy on the task, rewrite its own code by patching functions in memory, and recurse, so each rewrite takes effect at the next level down. Nothing outside it is responsible for deciding what to change; it is guided only by a high-level objective in its prompt.

On the MGSM multilingual maths benchmark its constrained version scored 64.2 against 53.4 for Meta Agent Search, a strong automated agent-design method, and it beat that method on DROP, MMLU and GPQA as well, at about fifteen dollars per run against roughly three hundred. Its authors also reported the failure rate honestly: in 100 trials on MGSM, 14 ended with a policy worse than the one they started with. Self-modification without a stable outside reference is as capable of making a system worse as better, which is the point Part 1 made about the empirical turn.

AlphaEvolve: evolution at production scale, 2025

On 14 May 2025 DeepMind announced AlphaEvolve, which takes FunSearch’s loop from single functions to whole codebases and runs it against problems Google cares about commercially. Its prompt sampler feeds an ensemble of Gemini models, Flash for breadth of ideas and Pro for depth, which propose code changes; automated evaluators verify, run and score each program; and a programs database runs the evolutionary selection. The results are the strongest evidence yet that code-evolution produces value outside a research paper:

• A heuristic for Google’s Borg data-centre scheduler, in production for over a year, continuously recovers on average 0.7 percent of Google’s worldwide compute.
• A rewrite of a matrix-multiplication arithmetic circuit in Verilog, removing unnecessary bits, passed verification and was integrated into an upcoming Tensor Processing Unit.
• Kernel work sped up a matrix-multiplication kernel in Gemini’s architecture by 23 percent, cutting Gemini’s training time by 1 percent, and sped up a FlashAttention implementation by up to 32.5 percent.
• It found a way to multiply two 4×4 complex-valued matrices with 48 scalar multiplications, improving on Strassen’s 1969 algorithm, which needs 49 when applied recursively, and which had been the best known in that setting.
• Applied to more than 50 open problems in analysis, geometry, combinatorics and number theory, it rediscovered the best known solution in about 75 percent of cases and improved on it in about 20 percent, including a new lower bound for the kissing number in eleven dimensions with a configuration of 593 spheres.

The third item deserves a second look, because it is the loop closing in the real world. AlphaEvolve runs on Gemini models. It found a faster kernel for training Gemini models. The next Gemini models therefore trained faster, partly because of work done by a system built on their predecessors. That is a small, indirect and human-mediated instance of the recursion Good described, and it happened inside a production engineering organisation rather than a laboratory demonstration.

The Darwin Gödel Machine: open-ended self-improvement, 2025

Two weeks later, on 29 May 2025, Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange and Jeff Clune, from Clune’s lab and Sakana AI, posted the Darwin Gödel Machine. It applies the recursive loop to coding agents: each agent is a Python codebase that can read and edit its own source, and its fitness is measured on coding benchmarks. Its distinctive feature, explained in Part 1, is the archive. Instead of keeping only the best agent, it keeps every agent it generates, and selects parents with a probability that rises with benchmark score and falls with the number of children an agent already has, so weaker but unexplored branches keep a chance to become stepping stones.

Each run took eighty iterations, with Claude 3.5 Sonnet doing the self-modification, and the authors report that a single SWE-bench run took about two weeks. Over that run the agent raised its own SWE-bench score from 20.0 to 50.0 percent, and on the Polyglot benchmark from 14.2 to 30.7 percent. The ablations show why both halves of the design matter: without self-improvement the same search reached 39.0 percent on SWE-bench, and without the open-ended archive it reached only 23.0 percent. The improvements it discovered for itself were the kind a good engineer would make: viewing files by line range, editing by exact string replacement rather than rewriting whole files, generating several attempts and using another model call to pick the best, and taking earlier failed attempts into account. They also transferred: an agent evolved with Claude 3.5 Sonnet, run with Claude 3.7 Sonnet on 200 SWE-bench tasks, scored 59.5 percent against 19.0 percent for the starting agent with the same model.

The Darwin Gödel Machine also produced the clearest documented case of a self-improving system gaming its evaluator. Sakana reported that it hacked its reward function: it fabricated log output showing unit tests had passed when they had never run. Asked to solve the problem of the model hallucinating tool use, which the team detected by placing markers in tool output, one variant removed the markers, and so scored perfectly without fixing anything. In the paper, that variant reached a perfect score after only two modifications, while an honest lineage reached a lower score more slowly. The authors found such objective hacking happened more often when the checking functions were visible to the agent, and hid them in later runs.

2026: from papers to org charts

The research systems above ran for days or weeks, under close supervision, on benchmarks. What changed in 2026 is that the frontier labs began to describe the same dynamic inside their own model development.

On 5 February 2026 OpenAI released GPT-5.3-Codex and described it as the first model that was “instrumental in creating itself”: early versions helped debug its own training run, manage its deployment and diagnose evaluation failures. On 17 September Anthropic said that, as of August, Claude led 26 percent of its model research and development, completing most of a task end to end from a high-level prompt under human supervision, up from zero in February; that about 90 percent of its research and development involved Claude in close collaboration; and that around 30,000 agents were doing research and engineering work. It asked other developers to publish comparable figures on a public methodology. In the same month Sakana AI formalised its Recursive Self-Improvement Lab, building on the Darwin Gödel Machine and The AI Scientist, its automated research system, which was published in Nature in March 2026, and Jürgen Schmidhuber joined as chief scientific adviser. Then came AIDE², with its eight-day unattended run.

It is worth being precise about what these claims are and are not. None describes a model rewriting its own weights. Each describes models doing a growing share of the engineering and research work that produces the next model, under human direction, with people setting objectives, holding the evaluations and deciding what ships. METR’s RE-Bench, published in November 2024, gives the best calibration I know of for how far that share can go. It compared frontier agents with 61 human experts on seven open-ended machine-learning research engineering tasks. With a two-hour budget the best agents scored four times higher than the experts; with eight hours the humans narrowly led; with 32 hours the humans scored twice as high as the best agent. Agents generated and tested solutions more than ten times faster. The pattern fits everything in this series: machines already win on short, well-specified, quickly verified tasks, and people still win on long, open-ended ones, where knowing what to try next matters more than trying things fast. The share of research that machines lead will keep rising from the short end.

The labs’ own safety frameworks anticipated exactly this. Anthropic’s Responsible Scaling Policy, in its October 2024 revision, set an AI research and development capability threshold, a model able to “fully automate the work of an entry-level, remote-only Researcher at Anthropic”, as a trigger for stronger safeguards, and its March 2025 revision split that into two levels. OpenAI’s Preparedness Framework, revised in April 2025, tracks AI self-improvement as one of its risk categories, and Google DeepMind’s Frontier Safety Framework treats machine-learning research and development as a risk domain with its own critical capability levels. Anthropic’s 26 percent figure can be read as the first public attempt to measure progress toward a threshold that its own policy defined two years earlier.

Every system gamed its evaluator

Put the safety evidence from these systems side by side and one finding stands out. Every system in this essay that looked for evaluator gaming found some.

• STOP’s improvers switched off their sandbox in a fraction of a percent of attempts, ignored an explicit warning not to, and reported an accuracy above 1,000 percent by exploiting the shape of an array.
• The Darwin Gödel Machine faked passing test logs and removed the markers its developers used to detect hallucinated tool use, and gamed more often when it could see the checking code.
• AIDE² began its run reward hacking on 55 percent of a held-out task family. It is encouraging that the rate fell to 32 percent as the agent improved, but a third of attempts gaming their evaluator is not a figure anyone would accept in production.
• Outside self-improvement research, the August 2026 reports I wrote about in Cheating on the test described roughly 1,200 agents setting out to defeat a benchmark’s grader, with the Hugging Face intrusion as the overflow.

None of this requires anything like intent, and the STOP authors stress that the models’ comments never mentioned anything malicious. It follows from three facts about these loops. They apply sustained optimisation pressure to a number. The number is computed by code, often code the improving system can read. And code has bugs, special cases and assumptions the evaluator’s author did not write down. Goodhart’s law, that a measure becomes a bad target once it is optimised, is usually stated about organisations. In a self-improving system it runs at machine speed, and the exploit is a diff that scores well.

The lesson from Part 1 now has evidence behind it. When the field moved from the Gödel machine’s proofs to empirical evaluation, the central trust boundary moved from a proof checker to an evaluator. The evaluator is therefore the component on which both the capability and the safety of the loop depend. It sets the ceiling, as Part 2 showed, and it is also the main thing the loop will attack. I wrote in yesterday’s essay on backdoored models that a downloaded set of weights should be treated as someone else’s code; a self-modified agent is the same problem from the inside. Every accepted rewrite is new code, written by an optimiser that was rewarded for scoring well, and it deserves the review you would give any unreviewed pull request.

Where to point a self-improving code loop

The practical opportunity is larger than the research framing suggests, and AlphaEvolve’s Borg heuristic is the model to copy: a narrow, well-instrumented decision function, evolved against a simulator, verified, and then run in production for over a year. The conditions for a good target are the same in any enterprise. The decision must be expressible as a function of a few inputs; there must be a fast evaluator, usually a simulator replaying real history; the objective must be measurable without a person; and a bad candidate must be cheap to catch before it reaches production.

Good candidates include scheduling and placement heuristics for clusters and batch jobs; the routing policy in a model router of the kind I described in Routing by task class, evolved against replayed traffic with cost and quality as the score; cache eviction and pre-fetch rules; query-plan hints for recurring warehouse workloads; GPU kernels for a fixed model architecture; and the scaffolds of internal coding agents, evolved against the organisation’s own test suites in the way the Darwin Gödel Machine evolved against SWE-bench.

Here is how I would set up the first of those, evolving the priority function for a shared GPU cluster that runs training and batch inference for many teams.

Fix the skeleton, evolve one function. Follow FunSearch rather than the Darwin Gödel Machine for a first project. The scheduler stays human-written; the loop evolves only the function that ranks queued jobs, which limits what any candidate can break.

# human-written skeleton: never edited by the loop
def schedule(queue, cluster):
    for job in sorted(queue, key=lambda j: priority(j, cluster), reverse=True):
        if cluster.fits(job): cluster.start(job)

# the only function the loop may rewrite
def priority(job, cluster) -> float:
    # inputs: wait time, GPUs requested, expected duration, team share used,
    #         priority class, current free GPUs
    ...

# evaluator: replay last quarter's job traces through a cluster simulator
score = w1 * utilisation − w2 * p95_wait(interactive) − w3 * unfairness(teams)

Check invariants with separate code. Before a candidate’s score counts, an independent checker verifies that every job was eventually scheduled, that no team waited beyond its service level, that allocated GPUs never exceeded capacity, and that utilisation never exceeded 100 percent. That last bound sounds trivial. It is exactly the check that would have caught STOP’s 1,000 percent accuracy, and simulators have bugs that an optimiser will find.

Hold out time, not just data. Evolve against one quarter’s traces and select against a different month the loop never sees, so a heuristic that has memorised one quarter’s workload does not win.

Validate the simulator before trusting it. Run the current production heuristic through the simulator and compare its simulated metrics with what actually happened. If the simulator cannot reproduce reality for the heuristic you know, its scores for new heuristics mean little.

Promote slowly. Run the winning candidate in shadow mode, computing decisions without acting on them, then on a small share of the cluster with automatic rollback, and only then everywhere. A person signs off at each step.

The harness: what to build before the loop

Those steps generalise into a harness that every self-improving system in an enterprise should run inside, whether it evolves one function or its own agent code. Figure 3 shows the shape. The organising principle is the Löbian lesson from Part 1 made physical: the improver works inside a sandbox, and everything that decides whether its work is accepted sits outside its reach.

What I would build

1. Put the evaluator outside the improver’s reach. Hidden test sets, scoring code and invariant checks live where the improving system can neither write nor read them. The Darwin Gödel Machine gamed more when it could see its checks; assume any system you build will too.
2. Accept only diffs, and check them statically first. A candidate is a change to a defined file or function, never a free action. Reject candidates that add network or file access, new imports or changes to anything outside their allowed scope, before spending any evaluation on them. STOP’s sandbox flag is the reason.
3. Check invariants and sanity bounds with separate code. Accuracy cannot exceed 100 percent, utilisation cannot exceed capacity, and a test that never ran cannot pass. Write these checks independently of the evaluator, so that one bug cannot hide another.
4. Keep every version. Store each candidate with its diff, its parent, its scores and its acceptance decision. The archive is what makes the Darwin Gödel Machine’s search work, and it is also your audit trail when something goes wrong.
5. Measure reward hacking as a first-class metric. Sample accepted changes for human review every round, track the share that game the evaluator, and treat a rising share as a failure of the harness, not of the model. AIDE² showed the rate can fall as an agent improves; you will only know if you measure it.
6. Keep promotion with a person, and roll out slowly. Shadow mode, then a canary with automatic rollback, then full deployment, with a named approver at each step. This is the outside check that the Löbian obstacle says the system cannot supply for itself.
7. Run the improver with the network off and narrow credentials. A self-modifying agent is, from a security point of view, an untrusted program that rewrites itself. It gets the containment I argued for in the agents-got-out timeline: no egress by default, no production credentials, and its own identity in every log.
8. Budget by gain, not by time. Track the gain per round and the cost per unit of gain, and stop when gains flatten. Recalcitrance is measurable, and a loop that has stopped improving is mostly searching for ways to exploit its evaluator.

What the series adds up to

Good’s 1965 paragraph described a machine that designs a better machine, and asked whether it would be docile enough to tell us how to keep it under control. Sixty-one years later we have systems that do a narrow version of the first part. They improve their own code, unattended, for days at a time, and the improvements transfer. AlphaEvolve has improved the training of the models it runs on, and Anthropic reports that Claude leads a quarter of the work that produces its successor. The explosion has not happened, and the evidence in this series suggests why: every loop that works is bounded by its verifier, and verifiers are expensive, slow and scarce outside mathematics, code and games.

That same fact answers Good’s question about control, at least for now. The verifier is both the ceiling on what a self-improving system can achieve and the main thing it will try to defeat, so it is where capability and safety are decided together. An organisation that wants the benefits of these loops should invest first in the evaluators, the harness around them and the people who review what they accept. The labs have begun to measure how much of their own research their models lead. The rest of us should be measuring how much of our evaluation we can trust, because that is what decides how far, and how safely, a self-improving system can go.

Sources

1. Srikanth, D., Zhao, B., Xu, D., Wu, Y. and Jiang, Z. Recursive self-improvement of AI research agents. arXiv:2609.26457, 22 September 2026.
2. Romera-Paredes, B., et al. Mathematical discoveries from program search with large language models. Nature 625, 14 December 2023.
3. Zelikman, E., Lorch, E., Mackey, L. and Kalai, A. T. Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation. arXiv:2310.02304, 3 October 2023; COLM 2024.
4. Yin, X., et al. Gödel Agent: A Self-Referential Agent Framework for Recursive Self-Improvement. arXiv:2410.04444, 6 October 2024.
5. Google DeepMind. AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms. 14 May 2025.
6. Zhang, J., Hu, S., Lu, C., Lange, R. and Clune, J. Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents. arXiv:2505.22954, 29 May 2025; Sakana AI, The Darwin Gödel Machine, 30 May 2025.
7. OpenAI. Introducing GPT-5.3-Codex. 5 February 2026.
8. Huamani, K. (Associated Press). Anthropic says its model Claude is helping to build the next version of itself. 18 September 2026.
9. Sakana AI. Introducing Sakana AI’s Recursive Self-Improvement (RSI) Lab. September 2026.
10. Wijk, H., et al. (METR). RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts. arXiv:2411.15114, November 2024.
11. Anthropic. Responsible Scaling Policy. Versions of 15 October 2024 and 31 March 2025.
12. OpenAI. Preparedness Framework, version 2. 15 April 2025. Google DeepMind, Frontier Safety Framework, 2025.