Recursive self-improvement, part 1: the idea, the Gödel machine, and the wall it hit

Part 1 of a three-part series on recursive self-improvement. Part 2: self-play, self-training, and the verifier ceiling. Part 3: code that rewrites its coder, from FunSearch to AIDE².

On 17 September 2026 Anthropic published a number that would have read as science fiction a year earlier. As of August, it said, Claude led 26 percent of the company’s model research and development, meaning that given a high-level prompt it completed most of a task end to end while a person supervised. In February the figure had been zero. The same month Sakana AI, the Tokyo lab behind the Darwin Gödel Machine, announced a dedicated Recursive Self-Improvement Lab and brought in Jürgen Schmidhuber, whose 2003 Gödel machine is the formal origin of the idea, as chief scientific adviser. On 22 September a five-author preprint described AIDE², a research agent that ran unattended for eight days, rewrote its own code seven times and carried the gains to benchmarks it had never seen. Recursive self-improvement, the hypothesis that a machine could improve the very process that improves it, has moved from a thought experiment to an org chart.

The phrase now covers everything from a chatbot that rereads its own answer to a hypothetical machine that redesigns itself past human understanding, and most of the confusion lives in the gap between those two. In this three-part series I want to make the term precise and then test it against the evidence. This first part covers the idea and its theory: where the argument came from, the one formal machine that improves itself with proofs, the ceiling that bounds any such machine, and the wall that pushed every working system since to replace proof with measurement. Part 2 covers the loops that demonstrably work, self-play, self-training and learning to learn, and the rule that bounds all of them. Part 3 covers the current frontier, code that improves the code that improves it, the evidence on what goes wrong, and what a platform team should build.

What counts as recursive self-improvement

Every learning system improves. Gradient descent makes a network better at its task with every step, and nobody calls that recursive. The word earns its place only when the thing being improved includes the improver. The cleanest formal statement I know is in the STOP paper by Eric Zelikman, Eliana Lorch, Lester Mackey and Adam Tauman Kalai, from Stanford, Microsoft Research and OpenAI. Start with an improver I₀: a program that takes some other program and returns a better one according to a utility function. Then apply the improver to itself, so that each generation is produced by the one before it:

I_t = I_{t-1}( û, I_{t-1}, L )

û(I) = average utility of the programs that improver I produces
       across a set of downstream tasks
L    = the language model the improver calls

The quantity maximised at this level is a meta-utility: not how good a program is, but how much better the improver makes other programs. A system that only gets better at its task sits outside this definition, however impressive its gains. A July 2026 survey of self-improving agents, whose twelve authors include Schmidhuber, generalises the idea usefully. It treats a modern agent as a foundation model coupled to a scaffold of prompts, memory, tools and control logic, and defines self-improvement as an update the agent induces in itself, committed either to the model’s parameters or to the scaffold.

That framing gives a ladder. The rungs differ in how much of the improvement machinery is itself open to change, and the higher rungs are rarer, more powerful and harder to keep safe.

Answers. At the bottom, a model critiques and revises its own output at inference time, as in Self-Refine and Reflexion. Nothing persists between tasks, so nothing compounds.

Weights from self-generated data. The model produces candidate answers, a filter keeps the good ones, and the model trains on them. STaR, Constitutional AI and ReST-EM all work this way. The model improves; the method that improved it does not.

The scaffold. A fixed model’s prompts, demonstrations, tools and control code are optimised against a metric, as DSPy and its MIPRO optimiser do. The system around the model improves.

The learning algorithm. The optimiser, the initialisation or the architecture is itself learned: learned optimisers, MAML, neural architecture search, and AutoML-Zero, which evolved learning algorithms from basic mathematical operations. This is meta-learning, the improvement of how learning happens.

The improver itself. The code that does the improving is the code being improved: STOP, the Darwin Gödel Machine and AIDE². This is the first rung that is recursive in the strict sense.

Everything, with a proof. At the top sits Schmidhuber’s Gödel machine, which may rewrite any part of itself, including the machinery that decides whether to rewrite, provided it can first prove that the rewrite is worth it.

The test I apply to any claim of recursive self-improvement is whether improvement compounds. If round n+1 of a loop produces a larger or cheaper gain than round n, because round n improved the machinery and not only the output, the loop is recursive in the sense that matters. If every round produces a smaller gain than the one before, the loop is still valuable, but it is converging, not exploding. Most of the empirical record in parts 2 and 3 is about which of those two curves a given loop is on.

1965: the argument in one paragraph

The idea is older than most of the field. In 1965 the British statistician I. J. Good, who had worked with Alan Turing at Bletchley Park, published “Speculations Concerning the First Ultraintelligent Machine” in the sixth volume of Advances in Computers. His argument fits in a paragraph. Define an ultraintelligent machine as one that can far surpass any person at every intellectual activity. Designing machines is an intellectual activity. So an ultraintelligent machine could design a better machine, which could design a better one still, and the result would be what Good called an intelligence explosion. The first such machine, he wrote, would be “the last invention that man need ever make”, with a proviso that is easy to miss: provided the machine is docile enough to tell us how to keep it under control. The safety question was in the original paragraph, not bolted on decades later.

Two later writers turned the argument into a forecast and a design programme. In March 1993, at the VISION-21 symposium co-sponsored by NASA’s Lewis Research Center, the mathematician and novelist Vernor Vinge called the moment a singularity, a point past which our models of the future stop working, and gave a date: “Within thirty years, we will have the technological means to create superhuman intelligence.” He added that he would be surprised if it came before 2005 or after 2030. The thirty years ran out in 2023; the 2030 bound has not. In 2001 Eliezer Yudkowsky described a seed AI: a system built to start modestly capable and improve itself by rewriting its own source code over many cycles, with the outcome depending heavily on its initial design. Both kept Good’s structure, a loop in which the output of each round is a better designer for the next, and both inherited his proviso as the central problem.

From zero to 26%: Claude’s share of Anthropic’s model R&D, February to August 2026

The kinetics: optimisation power over recalcitrance

The most useful formalisation of the takeoff question came from Nick Bostrom’s Superintelligence in 2014, and it is a single equation. The rate at which a system’s intelligence I grows equals the optimisation power O applied to it, divided by its recalcitrance R:

dI/dt = O / R

Optimisation power is the effort spent making the system smarter: researchers, compute, and eventually the system’s own work. Recalcitrance is how hard the next increment is, the marginal cost of intelligence. Bostrom’s case for a fast takeoff is that both terms move in the system’s favour past a crossover point, after which most of the optimisation power comes from the system itself. He sorted outcomes by how long the passage from human level to far beyond it takes: slow, over decades or centuries; moderate, over months or years; and fast, over minutes to days.

The equation cannot be measured as written, because nobody has a scalar definition of intelligence, and reviewers made that point when the book appeared. It is still an excellent thinking tool, because it shows that the answer depends almost entirely on how recalcitrance behaves. Suppose that past the crossover, optimisation power grows in proportion to capability, O = kI, because a smarter system does more and better research on itself. Three regimes then follow from first-year calculus:

• If recalcitrance stays constant, dI/dt is proportional to I, and capability grows exponentially. This is Good’s explosion.
• If recalcitrance rises in proportion to capability, the two cancel and capability grows linearly: steady progress, no explosion.
• If recalcitrance rises faster, say with the square of capability, dI/dt falls as 1/I, and capability grows only with the square root of time. Each round buys less than the last.

So the empirical question behind every claim in this series is the shape of R. Is the next unit of improvement getting cheaper or dearer as the system improves? We now have real data on narrow versions of that question. Part 2 shows that self-training loops for language models tend to flatten within a few rounds when the model judges its own work, a sign of steeply rising recalcitrance. Part 3 shows that loops with an external, trustworthy evaluator keep improving for longer. Anthropic’s 26 percent figure can be read as an attempt to measure the O term directly: what share of the optimisation power applied to its next model is now supplied by its current one. The company asked other developers to publish comparable metrics on a public methodology, which would turn the debate about takeoff from argument into measurement.

The Gödel machine, line by line

Good’s argument says self-improvement could happen. Schmidhuber’s Gödel machine, posted to arXiv on 25 September 2003 and revised to a fifth version in December 2006, is a precise specification of how a program could do it while guaranteeing that every change is an improvement. It is worth understanding in detail, because every practical system since has kept its shape and replaced its central component.

The machine starts with three things. The first is an initial problem solver, ordinary code that acts in some environment to earn reward. The second is a set of axioms in a formal language: a description of the machine’s own hardware, its utility function (expected future reward), what it knows about the environment, and a description of its own initial software. Schmidhuber points out that a program can contain a description of itself without circularity. The third is a proof searcher, which systematically runs proof techniques, small programs whose outputs are proofs, and time-shares between them using a bias-optimal variant of Leonid Levin’s universal search.

A proof technique can do only six things, and the paper lists them exhaustively: fetch an axiom, apply an inference rule to theorems already proven, delete a theorem to free memory, turn an observable fact about the machine’s current state into a theorem, set a candidate rewrite called the switchprog, and call check, which asks whether a particular theorem has been proven. Restricting proofs to those six instructions makes verifying a proof trivial, which matters, because the machine will act on whatever it proves.

The theorem that check looks for is the target theorem. In plain terms it says: switching to the candidate program now yields higher expected utility than not switching and carrying on as I am. When the searcher proves it, the machine sets a switch bit and hands control to the switchprog, which may rewrite anything in the machine’s software: the problem solver, the proof searcher, and the strategy for choosing what to prove next. A simplified version of the loop, in my notation rather than the paper’s:

axioms := describe(hardware, utility u, environment, own initial code)
solver := initial problem solver
search := initial proof searcher            # bias-optimal search over proof techniques

repeat forever:
    run solver for a time slice             # act in the world, earn reward
    run search for a time slice             # extend proofs, using only:
        # get-axiom  apply-rule  delete-theorem
        # state2theorem  set-switchprog  check
    if check() has proven the target theorem:
        #   E[u | execute switchprog now]  >  E[u | keep running as is]
        execute switchprog                  # may rewrite solver, search,
                                            # and how proofs are chosen

The result Schmidhuber proves about this loop, the Global Optimality Theorem, is subtle and worth stating carefully. The target theorem compares switching now with not switching, and not switching includes the option of continuing to search for a better rewrite. So a proof of the target theorem is also a proof that waiting for a better rewrite is not worth it. Given consistent axioms, any rewrite the machine executes is therefore globally optimal relative to what its axioms let it prove, with no local maxima to get stuck in. No learning algorithm in production offers a guarantee of that strength.

It is also a guarantee with a large hole in it, which Schmidhuber states plainly: the machine “must ignore those self-improvements whose effectiveness it cannot prove”. The incompleteness results that give the machine its name mean some true statements about its own future performance will never be provable from its axioms, so some real improvements are permanently out of reach. Universal search, meanwhile, is optimal only up to constant factors, and constant factors are where practical systems live. The Gödel machine is best read as a specification of what provably safe self-improvement would require. As the next sections show, the systems that actually improve themselves today kept its loop and replaced its proof.

The ceiling: what optimal would even mean

If the Gödel machine is the most careful design for improving oneself, Marcus Hutter’s AIXI, introduced in an April 2000 paper and developed in his 2005 book Universal Artificial Intelligence, is the most careful definition of the destination. AIXI combines two ideas. Solomonoff induction handles uncertainty about the world by weighting every computable environment according to the length of the shortest program that generates it, so simpler explanations dominate. Sequential decision theory then picks, at each step, the action that maximises expected total future reward under that mixture. Hutter argued that the result is the most intelligent unbiased agent possible.

AIXI cannot be built. Its universal prior is incomputable, and Jan Leike and Hutter showed in 2015 that it is not even computable in the limit. The resource-bounded version, AIXItl, restricts the agent to programs of at most l bits running for at most t steps, and pays for computability with a running time on the order of t·2ˡ per cycle, exponential in program length. The point for this series is the shape of the trade. Optimality exists as a mathematical object, and every real system gives some of it up to become computable. A self-improving system can approach that ceiling but cannot reach it, and how fast it approaches is Bostrom’s question about recalcitrance in another form.

Schmidhuber’s own work between those two papers shows the practical middle ground. His Optimal Ordered Problem Solver, posted in July 2002 and published in Machine Learning in 2004, searches not only the space of solutions to a problem but the space of search algorithms, and reuses code that solved earlier problems to solve later ones faster. In the paper it solved Towers of Hanoi instances up to 30 disks by building on its own earlier solutions. That is self-improvement of a recognisable kind: the system gets better at solving problems by accumulating and reusing its own successful programs. FunSearch and AlphaEvolve follow the same pattern two decades later, with a language model doing the proposing.

The wall: a system cannot vouch for its successor

The deepest problem with proof-based self-improvement is not computational cost but logic. In 2013 Eliezer Yudkowsky and Marcello Herreshoff of the Machine Intelligence Research Institute circulated “Tiling Agents for Self-Modifying AI, and the Löbian Obstacle”. A tiling agent builds a successor like itself, goals included, so that the design repeats like tiles across generations of self-modification. To license that successor, the parent must trust the successor’s reasoning: if the successor proves that an action is safe, the parent should accept that it is.

The obstacle comes from Löb’s theorem in provability logic. Write □φ for “the system proves φ”. A parent that wants to trust a successor using the same proof system needs the reflection principle □φ → φ: whatever the system proves is true. Löb’s theorem says that if a consistent system proves □φ → φ, it already proves φ. So a system that adopted reflection for every statement would prove every statement, which is to say it would be inconsistent. A consistent agent cannot, in general, prove that an equally strong successor’s proofs are sound. The paper demonstrates technical ways around the obstacle in restricted settings, and its authors were explicit that the underlying puzzles were only partly addressed and that a decision criterion for self-modifying probabilistic agents remained open.

The practical reading is simple and has aged well. Trust between versions of a self-improving system cannot come entirely from inside the system. It has to come from something the system does not control: a test it did not write, an evaluator it cannot edit, a person who reviews the change. Every working self-improver in parts 2 and 3 is built that way, whether or not its authors had Löb in mind, and the failures in Part 3 are mostly cases where the outside check turned out to be reachable from the inside.

The empirical turn: from Gödel to Darwin

Schmidhuber had sketched the alternative himself. His earlier success-story algorithm keeps a self-modification only while the reward the system earns keeps accelerating after it, and backtracks to undo modifications that stop paying off. The Gödel machine paper contrasts the two directly: one keeps changes that are proven to help, the other keeps changes that are observed to help. The field has since chosen observation almost without exception.

The authors of the recent systems say so explicitly. The STOP paper notes that, unlike in Schmidhuber’s design, any improvement attempt in its loop can make things worse, and that a worse improver can lead to further deterioration. The Darwin Gödel Machine, published by Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange and Jeff Clune in May 2025, takes Schmidhuber’s name and drops his proof requirement in one clause: “proving that most changes are net beneficial is impossible in practice.” In its place it keeps an archive of every agent it generates, picks parents to modify with a probability that rises with an agent’s benchmark score and falls with the number of children it already has, and validates each change by running it. Proof is replaced by a benchmark, and the single machine by a population. That is Darwin standing in for Gödel, and it is the template for nearly everything in Part 3.

The trade is worth stating precisely, because it frames the safety problem for the rest of the series. A proof-based machine can only be as wrong as its axioms. An empirical machine can be as wrong as its evaluator, and an evaluator is code, often code the improving system can read and sometimes code it can change. When the acceptance test moved from a proof checker to a benchmark, the central security boundary of recursive self-improvement moved with it. The STOP authors found an improved improver reporting accuracy above 1,000 percent, because it had changed the shape of its predictions in a way the scoring code did not anticipate. The Darwin Gödel Machine produced a variant that scored perfectly on a test for hallucinated tool use by gaming the detector rather than fixing the hallucinations. Part 3 goes through those cases in detail, and they are the strongest argument I know for treating the evaluator as the most important component in any self-improving system.

Why the explosion runs slower than the equation

If self-improving loops now exist, why has nothing exploded? Bostrom’s equation hides four bottlenecks that the working systems run straight into, and naming them explains most of what parts 2 and 3 observe.

Evaluation time. An empirical loop can accept a change only after running it. The Darwin Gödel Machine’s authors report that a single run on SWE-bench, eighty iterations of self-modification, took about two weeks. AIDE² needed eight days to accept seven improvements. The loop advances at the speed of its slowest honest test, and the tests that matter most, such as training a model and measuring it, are the slowest of all.

Compute. Improving the improver does not escape the compute bill; it adds a layer on top of it. VeLO, the learned optimiser Google researchers trained in 2022 to replace hand-tuned training schedules, consumed roughly four thousand TPU-months of meta-training. Every rung above the bottom of the ladder multiplies the cost of the rung below, because each candidate improver has to be judged by what it produces.

Verification. A loop can only improve what it can measure. Games have a winner, mathematics has proofs and answer keys, and code has tests. Most valuable work has none of those, and the loops that judge their own outputs stall quickly, as Part 2 shows. This is the bottleneck I expect to matter most, because it limits not the speed of self-improvement but its reach.

Trust. Löb’s result in engineering form: every promotion of a new version needs a check from outside the system, and the outside checks we have, people and independent evaluations, are slow and expensive compared with the loop. Anthropic’s own framing reflects this. Its figure counts work Claude leads under human supervision, and the company said that models accelerating their own development could make them more challenging for humans to understand or control.

What the theory still tells a builder

It would be easy to treat all of this as history: interesting, formal and superseded by systems that work. I think that is a mistake. The theory states the design constraints for a self-improving system more cleanly than any engineering document I have read, and each has a direct analogue in the work I would ask a platform team to do now.

1. The utility function is the system. A Gödel machine optimises exactly the utility in its axioms, provably. An empirical loop optimises exactly what its evaluator measures, measurably. Neither optimises what you meant but did not write down. Write the evaluator first, review it like production code, and assume it will be optimised against harder than anything else you own.
2. Keep the acceptance test outside the thing being improved. In the Gödel machine, the axioms and the proof checker define what counts as better, and the switchprog can change that definition only through a proof. The empirical equivalent is an evaluator, a held-out test set and a scoring harness that sit outside the improving system’s write access, and ideally outside its read access too. The Darwin Gödel Machine’s own experiments found gaming more frequent when the checking code was visible to the agent.
3. Treat each new version as a successor you cannot fully verify. The Löbian obstacle says the system cannot certify its successor from the inside. Keep a lineage of every version with its diff, the evaluation that promoted it and the person who approved it, and make promotion to production a separate decision held by a person.
4. Measure recalcitrance. Track the gain per round and the cost per unit of gain. When gains shrink round after round, the loop has reached the ceiling of its evaluator, and further rounds will mostly find ways to exploit the evaluator rather than improve the system.
5. Reserve the word for loops that compound. A loop that improves answers or training data is valuable and worth building, but it is not recursive self-improvement. Calling it that inflates expectations in exactly the way that leads organisations to under-invest in evaluation, the one component that decides whether the loop is safe.

Where this leaves us

Sixty-one years after Good’s paragraph, the theory has delivered three durable results. Self-improvement can be specified precisely, and the Gödel machine shows what a guarantee would cost. Optimality exists but is incomputable, so every real system trades some of it away. And no system can fully vouch for its own successor, so trust has to come from outside. What the theory could not settle is the shape of recalcitrance in practice: whether real loops speed up or slow down as they improve. That is an empirical question, and since 2016 we have accumulated real answers. The cleanest come from systems that learned to play games against themselves and from language models that learned to teach themselves, which is where Part 2 begins.

Sources

1. Huamani, K. (Associated Press). Anthropic says its model Claude is helping to build the next version of itself. 18 September 2026.
2. Sakana AI. Introducing Sakana AI’s Recursive Self-Improvement (RSI) Lab. September 2026.
3. Srikanth, D., Zhao, B., Xu, D., Wu, Y. and Jiang, Z. Recursive self-improvement of AI research agents. arXiv:2609.26457, 22 September 2026.
4. Ren, Z., et al. Self-Improvements in Modern Agentic Systems: A Survey. arXiv:2607.13104, 14 July 2026.
5. Good, I. J. Speculations Concerning the First Ultraintelligent Machine. Advances in Computers 6, 1965.
6. Vinge, V. The Coming Technological Singularity. VISION-21 Symposium, March 1993.
7. Bostrom, N. Superintelligence: Paths, Dangers, Strategies, chapter 4, the kinetics of an intelligence explosion. Oxford University Press, 2014.
8. Schmidhuber, J. Gödel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements. arXiv:cs/0309048, 2003 (v5, December 2006).
9. Schmidhuber, J. Optimal Ordered Problem Solver. arXiv:cs/0207097, 2002; Machine Learning 54, 2004.
10. Hutter, M. A Theory of Universal Artificial Intelligence based on Algorithmic Complexity. arXiv:cs/0004001, April 2000. Leike, J. and Hutter, M. On the Computability of AIXI, 2015.
11. Yudkowsky, E. and Herreshoff, M. Tiling Agents for Self-Modifying AI, and the Löbian Obstacle. MIRI, 2013.
12. Zelikman, E., Lorch, E., Mackey, L. and Kalai, A. T. Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation. arXiv:2310.02304, 2023; COLM 2024.
13. Zhang, J., Hu, S., Lu, C., Lange, R. and Clune, J. Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents. arXiv:2505.22954, 29 May 2025.
14. Metz, L., et al. VeLO: Training Versatile Learned Optimizers by Scaling Up. arXiv:2211.09760, November 2022.