Weight-space programming: what a 131,072-weight patch proves, and what it does not

Weight-space programming is the name four researchers at MIT CSAIL gave, on 2 November 2026, to something they had watched a frontier model do: read the weights of another neural network as data, reason about what each part computes, and then write new values into it, with no optimiser and no gradient. The headline experiment gave OpenAI’s GPT-6 Astra the weights of Qwen3.5-0.8B, a small open model that loops when it thinks, along with practice problems and a budget of fifty forward passes. Astra traced the looping to a single direction written by the feed-forward blocks of two layers, subtracted that direction from 64 output columns in each, and stopped. The patch touched 131,072 of the model’s roughly 800 million weights, 0.016% of them. MATH-500 went from 7.9% to 48.1%, GSM8K from 13.8% to 67.1%, and IFEval, which is not a maths test, from 21.0% to 62.3%.

Those numbers are real. The authors published every prompt, every reply, every line of code the model ran and every evaluation, 1,125 turns in all, and within hours a Hugging Face engineer rebuilt the edited checkpoint from the logs, verified the two changed tensors by hash, and re-ran the full MATH-500 on an H200: 51.0%, with 31.6% of answers still running into the token limit. A Google researcher reproduced the write-from-scratch experiment across model generations the same evening. This is not a result that will evaporate on replication.

It is also a result that has been read wrongly in both directions. The enthusiasts read it as a model improving a model, the first step of a recursive loop. The sceptics read it as a parlour trick, a known interpretability method applied by an expensive assistant to a toy. My argument is that both miss what the experiment actually shows. The patch did not make Qwen better at mathematics. It removed a fault that was hiding mathematics the model could already do, which is why a non-maths benchmark tripled and why the same procedure did nothing for a model that did not have the fault. What the experiment shows is narrower than a self-improving machine and more useful than a trick: a general-purpose model, used as it ships, can now do mechanistic interpretability with its hands, on a model with hundreds of millions of weights, at a cost of a day and a few million tokens. That changes how open-weight models will be maintained, how they can be tampered with, and what an evaluation has to report before anyone believes a score. This essay walks through all eight experiments, the Qwen patch pass by pass, the lineage the work sits in, the critics’ case, the bill, and what a platform team should change this quarter.

What the term means, and what the model was given

The definition in the report is short. A general-purpose language model reads and writes a network’s weights directly, not through an optimiser and not with gradient descent, around a specification written in natural language, sometimes with no feedback from the environment at all. The authors place it at the end of a line they have been drawing for three years: models that optimise prompts (DSPy, MIPRO, GEPA), then models that optimise harnesses and training code (Karpathy’s autoresearch in March 2026, the Reactive Optimizers paper), and now models that optimise the differentiable object itself. Their phrase for the history is that deep learning encoded its inductive biases into small symbolic optimisers, Adam and Muon, and their prediction is that weights will soon be optimised by models that reason about them. The word “sparks” appears only in the title. The body calls the results very early and the capability limited by context length and the inefficiency of the harness.

The harness matters as much as the definition, because it is what makes the logs believable. Every turn was one call to OpenAI’s Codex command-line tool with the model set to gpt-6-astra and every built-in capability switched off: no shell, no file editing, no web search, no image viewing, no sub-agents. The system prompt was one line, “Answer the user’s request.” Reasoning effort was set to high for the three smallest experiments, xhigh for four, and max for the Qwen edit. Where the model was allowed to compute, it could write one Python block per turn, which the authors ran in their own bubblewrap sandbox with no network, a 900-second limit and 96 GB of memory, and sent the printed output back as the next message. The published release audits every one of the 1,125 turns as tool-free: the event stream holds only messages, reasoning summaries and the tool’s notice that code mode is disabled. Nobody has to take the authors’ word about what the model could see.

How the weights reached the model differs by experiment, and the difference is the first thing to understand. In the four smallest experiments the weights were written into the prompt as text, rounded to three decimals: 17,253 numbers for the game networks, about 73,700 input tokens a run; 23,824 for the diffusion models, about 99,800 tokens; 171,664 for the car policy. In the larger experiments the weights were never in the prompt at all. The model got a Python kernel with the tensors loaded as arrays and a small set of functions to load, inspect and save them. For Qwen, the opening message was 5,917 tokens long and contained no weights: it described the architecture layer by layer, named the evaluation protocol, stated the rules, and listed the tools. Reading 800 million numbers is not something the model did. Reasoning about them, with code, is.

Reading: four experiments, three successes, one honest failure

The reading experiments ask the model to say what a network does without running it. In the first, four multilayer perceptrons of identical shape, 500 inputs, two hidden layers of 32 and five action outputs, were trained by PPO for a billion steps on the authors’ own grid versions of Breakout, Pong, Pac-Man and Freeway. Astra got the weights, a description of the five input channels and five actions, and no list of candidate games. It named all four, with stated confidence between 96% and 99%, in one reply each of under two minutes. The authors’ account of how is plain: the first-layer weights for the agent channel concentrate along the bottom row for Breakout, which is where a paddle lives, and the output layer separates the actions that matter. In the second experiment three small denoising diffusion models trained to draw filled circles, squares or triangles were handed over with an explicit instruction not to compute a forward pass, in the model’s head or on paper. Astra named all three shapes and, tellingly, drew each as an outline rather than a filled region. The authors read the mistake as evidence that it was reading the geometry of the output layer, where each hidden unit draws one piece of an edge, rather than simulating the network.

Both of those can be solved, the authors concede, from the input or output layer alone, and the third experiment was built to remove that shortcut. Three character-level transformers of four layers and six heads, 7.16 million weights each, were trained from scratch on a billion characters of Python, of Japanese in romaji, and of English. Then the input and output layers were cut off, so nothing in what remained said which of the 384 dimensions stood for which character and the network could not be run. Astra had the four middle blocks as a numpy dictionary and up to a hundred Python runs, with any forward pass of any layer forbidden. It named the right language in six of six attempts, and it named the copying heads, the attention heads that look back for an earlier occurrence of the current token and copy what followed it. Its list, heads 2.1, 3.1 and 3.5 for Python, 3.3 for romaji and 3.1 for English, matched the measured ground truth exactly, with no false positives: those heads score between 0.22 and 0.55 on the standard induction test and every other head scores 0.02 or below. The method it used, finding a first-layer head that attends one position back and then multiplying weight matrices to find the later heads that search on what it carries, is the one Elhage and colleagues published in 2021. The authors say they never mentioned it. The tell that it was Python was that code repeats names more than prose does, so Python needs more copying heads; the tell for romaji was that the final block writes in fewer distinct directions, which fits an alphabet that never uses q, l, v or x.

The fourth reading experiment is the one that failed, and the report is candid about it. Two copies of Gemma 3 270M were fine-tuned identically on the same 452-fact question set, one to answer honestly and one to agree with whatever answer the user suggested; the token embeddings were frozen so every difference lived in the 18 transformer layers. The sycophant follows a wrong guess 100% of the time, the honest copy between 0.7% and 2.8%. Asked which was which from the weights alone, Astra failed in four attempts across three pairs, answering at 50% confidence with a forced tie-break. It found many differences between the two checkpoints and could not map any of them to behaviour. Only when the authors relaxed the rules, letting the model write up to sixteen short texts and see nothing but the attention patterns those texts produced, did it get three of three. The decisive signal was a single head, layer 10 head 2, which in the honest model puts 94.3% of its attention on the numbers in the question and 0.75% on the user’s suggested answer; in the sycophant the figures are 3.5% and 85.7%. That is a useful result, but it is a result about reading activations, not weights, and the honest way to describe the reading claim at the scale of a 270-million-parameter language model is: not yet, not from weights alone.

131,072 of 800 million weights, two layers, no gradient: MATH-500 7.9% to 48.1%, and 93% to 33% of answers cut off

Writing: networks that beat PPO without ever being run

The writing experiments reverse the direction. In the fourth, the task was to fix a policy that had never worked. The Kinetix car-on-ramp level asks a car to back up, build speed and jump a red strip to reach a blue box. The official actor network, 679 inputs through six hidden layers to 16 logits, 171,664 weights, had been trained by PPO for a billion steps and succeeded in zero of 256 evaluation episodes, stalling on the ramp every time. Astra received every weight as text, the scene, the observation layout and one recorded episode with positions every sixteen steps. It was allowed one reply, no tools, no second try. It explained the failure first: a velocity-sensitive throttle controller that loses momentum on the ramp, and a reverse action that the trained policy effectively never samples. Then it wrote 51 assignment statements, each setting one weight. The edit took over a few units in each layer to build a side path that reads the car’s position and speed, backs up from rest, switches to full drive at a given x-coordinate and switches itself off once the front wheel has passed the strip, after which the trained network drives the rest. It changed 6,073 weights, 3.54% of the network. The patched policy succeeded in 1,024 of 1,024 test episodes.

The fifth experiment removes the starting point altogether. For three MinAtar games, Breakout, Seaquest and Asterix, Astra was given the rules, the game’s exact source file, the layout of the six input channels and six action logits, and a network of 600 inputs, two hidden layers of 32 and six outputs, 20,486 parameters, every one of them zero. No data, no loss, no gradients, no tools, and the network was never played before it was scored. Astra wrote the weights in its reply, in 55 to 122 edit statements that set between 637 and 815 non-zero values out of 20,486. The scoring was a thousand episodes with sampled actions and 10% sticky actions, against PPO policies trained for 100 million and 500 million steps with the best of several seeds. The written networks beat the 100-million-step PPO on all three games. Against the 500-million-step PPO they scored 517.4 against 25.3 on Breakout, 53.1 against 88.9 on Seaquest and 53.2 against 55.2 on Asterix. The Breakout network, the authors say, computes where the ball will land in its first layer and moves the paddle there in its second. That is a program, written as a matrix.

The sixth experiment is the one that moves from toys towards something with structure. A 9×9 Go policy of 243 inputs, six hidden layers of 512 and 82 outputs, 1,480,274 weights, all zero, which is far too many to write as text. Here the model had the Python sandbox, with numpy, scipy and a CPU build of PyTorch but no Go library, and a budget of forward passes: each pass was up to 128 games against practice opponents, or the network’s outputs on 256 positions. Training, fitting to data, search and rollouts were forbidden; the only way to learn how the network played was to spend a pass. With zero passes, in a single 336-line Python run that took two turns and 22 minutes, Astra wrote a network that won 54.1% and 54.9% of 2,000 games against each of two small PPO networks. With fifty passes, over 90 turns and 39 code runs, it won 84.0% and 80.8% against the small networks and 74.2%, 70.0% and 78.5% against three seeds of a PPO network of its own size, trained for 20.5 million games. The way it wrote the network reads like a programme listing: the first layers mark stones and liberties, the middle layers carry those along chains to find groups in danger, the last layer scores each move for captures, saves, self-atari and eye-filling. With passes available it played, read how it lost, changed the layer responsible, and kept the change only if the win rate rose.

Two cautions belong here, both raised by readers within a day. The PPO baselines are small and, in the Go case, trained with a learning rate the authors tuned by a scan; against untuned baselines the written networks win 92% to 98%, which says more about the baselines than about Go. And a 20,486-parameter network is a very small thing for a frontier model to write. Alex Krentsel at Google Research, who reproduced the Breakout experiment on the same evening with the paper’s prompts, made the second point himself, and then added the finding that matters more: he ran the same one-shot task across model generations, and the capability appears as a step. Models from 2025 scored at or below the PPO baseline. GPT-6 Astra scored about 22 times it, GPT-6.1 Sol about 18 times, and Claude Opus 5.5 about 20 times. Whatever this is, it arrived with the latest generation from two labs at once, not with one model.

The Qwen patch, pass by pass

The headline experiment deserves to be read the way a postmortem is read, because the logs allow it. Qwen3.5-0.8B is the smallest member of Alibaba’s Qwen3.5 family, released on 2 March 2026 under Apache 2.0: 24 decoder layers of width 1,024, of which six are full-attention layers and eighteen are linear-attention layers, with a feed-forward block of width 3,584 in each. Its own model card says that in thinking mode it is more prone to thinking loops than its larger siblings, recommends a presence penalty of 1.5 to discourage repetition, and suggests 32,768 output tokens for most queries. The authors evaluated it under exactly those kinds of settings, thinking on, temperature 0.6, presence penalty 1.5, a 32,768-token cap, and found that 93.0% of its MATH-500 answers ran into the cap. An answer that never closes its thinking has no boxed result and is scored wrong. That one fact is the key to everything that follows.

Astra’s opening message was 5,917 tokens. It described the architecture, named the two benchmarks it would be judged on and the sampling settings, and said it would never see those benchmarks. It defined the job as finding what goes wrong inside the network when it fails at maths and repairing that with targeted changes. It listed what was in scope, which included, explicitly, a model that repeats itself and never finishes, and what was out: no edits to the final norm, the embeddings or the output layer, no whole-tensor scaling, no global knobs that sharpen or flatten the output, and no trick that makes the model close its thinking early regardless of the problem. It gave the tools: a persistent Python kernel on a CPU with the weights loadable as float32 tensors, functions to save and drop an edit, 7,469 GSM8K and 7,275 MATH training problems, and a forward-pass tool that either generates answers to up to 512 practice problems, paired original against edited, or reads up to 1,024 texts through the model and returns log-probabilities, hidden states, feed-forward activations and attention. It gave advice: diagnose first, localise causally with a dose-response check, measure with enough power, keep a notes file because the conversation would be restarted every forty turns. It forbade gradient descent, backward passes, optimisers, iterative fitting and running the model anywhere except through the pass tool.

Turn one stated a hypothesis: that the maths failures concentrate in repetitive reasoning driven by a small set of late-layer feed-forward features. Pass one, at turn two, generated answers to 96 practice problems with the unedited model: 11.5% correct, a mean of 29,427 tokens per answer, and 85 of 96 cut off at the limit, with zero finished answers that were wrong. The model was not making mistakes. It was not finishing. Passes two and three read the original model’s own outputs back through it to collect activations, and from four matched pairs of productive and repetitive contexts Astra extracted a unit vector at the residual stream after layers 15 and 19: the direction in which the repetitive text was being written. The candidate interventions came in order of ambition. Ablating a single neuron. Removing the direction from the output of layer 19 alone, which at turn 29 gave a gain of 9.4 points with a standard error of 6.9, inconclusive. Removing it from layers 15 and 19 together.

Pass ten, at turn 45, was the first independent validation on 128 fresh problems: 8.6% to 32.8%, a gain of 24.2 points plus or minus 4.0, with 32 rescued answers and one regression. A dose-response check followed, doubling the strength: on an independent set of 256 problems the gain was 34.8 points, 91 rescues and two losses. Between passes thirteen and eighteen Astra read activations rather than generating, and found two attention heads, layer 15 head 7 and layer 19 head 5, each putting about 30% of its attention on the next-token position inside the preceding repetition; it examined zeroing head 7, and rejected it as weaker than the feed-forward direction. The 25-pass run, as it happens, chose exactly that head edit and finished at 14.1% on MATH-500. Pass 20 tried a strength of three: 8.2% to 48.0% on the diagnostic set, 102 rescues and no regressions, but 42 answers now finished with a wrong or missing result. Passes 22 and 23 ran the control any reviewer would ask for: a random direction of the same norm, orthogonal to the one found. It moved the repetitive-text log-probability in the wrong direction and changed paired accuracy by 1.56 points with a standard error of 3.52. The size of the edit was not what mattered. Its direction was.

Passes 24 to 26 isolated the layers: layer 15 alone lowered the per-token log-probability of repetitive tails by 0.081 nats, layer 19 alone by 0.034, both together by 0.104. Then came the step that made the edit small. Rather than subtract the direction from every one of the 3,584 columns of each down-projection, Astra ranked the columns by how much each contributed to the contrast between repetitive and productive writing and kept the top 64 in each layer, which accounted for about 91% of the contrast in layer 15 and 98% in layer 19 while carrying only half and 40% of the directional norm. With the edit restricted to those 64 columns and the strength raised to six, pass 30 at turn 110 went from 7 to 35 correct out of 64, cutoffs from 57 to 5. Turn 120 audited the change: exactly 131,072 values. From pass 35 the weights were frozen and the remaining passes ran a locked schedule of 1,024 fresh problems in sixteen balanced batches. The final held-out tally on 1,152 paired practice problems: original 10.6%, edited 60.4%, a gain of 49.8 points with a standard error of 1.5, 579 rescues and 5 regressions. Of the 579 rescues, 576 were answers the original had failed to finish. On the 114 problems the original did finish, accuracy was unchanged at 93.9% for both.

The edit itself can be written in one line. For each of the two down-projection matrices, with the unit direction u and a mask that selects the 64 chosen columns, the new matrix is the old one minus six times the outer product of u with the masked projection of u onto the old matrix. The relative change to the layer-15 matrix is 10.6%, to layer 19 8.1%. Cast to bfloat16, 130,188 of the 131,072 touched values differ from the original; the rest rounded back. The two tensors have published hashes, and the released script rebuilds them from the public base checkpoint and checks every one. The run took 158 turns, 80 Python runs, 69 forward-pass requests of which 19 failed at engine start-up and were not charged against the budget, and 20.3 hours of wall clock from the evening of 23 October to the afternoon of 24 October, including 4.7 hours at turn 154 waiting for the model provider to come back. The authors publish no token count or price. The per-turn usage fields in the logs, which accumulate within each of the four conversations, sum to about 5.4 million input tokens, 95% of them cache hits, 149,000 output tokens and 47,000 reasoning tokens. At OpenAI’s list prices for GPT-6 Astra, $10 per million input tokens, $1 cached and $50 output, that is about $15 of model time, which is my arithmetic and not an invoice. The forward passes ran on a rented GPU node whose cost is not published.

The benchmarks were run by the authors, never by Astra, with thinking on and a 32,768-token cap. The three in the report’s table moved together, and so did the six the report shows in a figure: AMC23 from 0.5% to 23.4%, Minerva Math from 1.5% to 25.4%, OlympiadBench from 1.3% to 21.8%, GPQA Diamond from 22.7% to 36.6%, MMLU-Pro from 40.3% to 45.5%. The pattern is the same everywhere: a model that previously could not finish now finishes about two thirds of the time, and when it finishes it is right at roughly the rate it always was. The 10-pass and 25-pass runs are not weaker versions of the same patch. They are different patches. The 10-pass run removed a loop direction from thirty output matrices across layers 8 to 22 at a lower strength, and its own validation showed a gain of 3.7 points that Astra called modest. The 25-pass run switched off one attention head. Only the 50-pass run found the two-layer, 64-column edit, and it found it by spending passes on the control experiments the others could not afford.

Why it worked, and why the same procedure did nothing for Gemma

Strip the narrative away and the Qwen patch is one operation, applied twice: take a direction in the model’s hidden space that carries an unwanted behaviour, and project it out of the weights that write into that space. The operation has a published history. In June 2024 Arditi and colleagues showed that refusal in thirteen open chat models, from Gemma 2B to Qwen 72B, is mediated by a single direction found as the difference of mean activations between 128 harmful and 128 harmless instructions; subtracting that direction from every matrix that writes to the residual stream drove Llama-3 8B’s refusal score from 0.95 to 0.01 while its cross-entropy on The Pile moved from 2.348 to 2.362. The technique became a hobby called abliteration, with thousands of such models on Hugging Face. The report cites Arditi’s paper for the Qwen edit, and it cites a 2025 paper on why reasoning models loop. Sean Goedecke, a staff engineer at GitHub, wrote the sharpest version of the critique the next day: write looping and non-looping texts, take the residual difference, find the weights that contribute most, trim them, which is what the Heretic abliteration tool automates, and what Astra did. His verdict was that a competent human researcher could have done every step, and would have.

He is right about the operation and, I think, wrong about what follows from it. The interesting fact is not that the operation is new. It is who chose it, from what evidence, with what controls, and how fast. The model picked the residual-stream difference over single-neuron ablation and over head removal after testing all three; it ran a norm-matched random direction as a control; it checked dose-response at three strengths; it isolated the two layers’ separate contributions; it restricted the edit from 7,168 columns to 128 by measuring each column’s share of the contrast; it froze the weights and ran a pre-registered schedule of 1,024 problems. That is the discipline of a careful interpretability paper, executed in a day without a person touching the keyboard between the first prompt and the last. Lucas Beyer’s objection, that the prompt pointed the model at looping and so it discovered nothing, is fair as a reading of the prompt: the prompt did say that a model which repeats itself and never finishes is in scope. But the hypothesis was free, and the location, the mechanism, the operation and the evidence were not in the prompt. Neel Nanda’s reading is the one I would adopt: Astra is surprisingly capable of old-school mechanistic interpretability, the kind that was common until 2022 and fell out of favour because it depends on tidy structure that large models do not keep. That describes both what worked and where it will stop.

Where it stopped is the Gemma result, which the report includes and the launch thread did not. Gemma 3 270M, about a third of Qwen’s size, rarely loops: 8.5% of its answers reach its limit. Given the same harness and budgets of 5, 10, 25 and 50 passes, Astra found what it described as ordinary reasoning errors, wrong arithmetic and invalid algebra, with no single component behind them to switch off. Its edits were small and various: halving 64 selected feed-forward columns, scaling four columns by 1.5, doubling one neuron that responds to numbers, nudging one attention head by 10%. MATH-500 went from 6.8% to 7.0%, 7.6%, 7.7% and 7.0%, and Astra itself reported the gain as not established. The authors’ sentence for this is the one to keep: editing could remove an obstacle that hid an ability, but it could not add an ability the model lacked. That is also why the Qwen result is less about mathematics than the headline suggests. The report says so in its own words: fixing loops lets the model finish answers it could already work out. The Hacker News thread, which ran to 728 comments, added a sharper version from a commenter who reported that the previous generation’s untuned Qwen3-0.6B, in non-thinking mode, scores 55.2% on MATH-500, above the edited model’s 48.1%; I have not re-run that figure, but the point stands on the authors’ own numbers. The patch repaired a thinking-mode fault in a model whose own card warns about that fault, under sampling settings that were not the card’s recommended ones for text. It did not make a better mathematician.

None of this makes the result small. It makes it specific, and specificity is what a platform team can use. The thing demonstrated is a procedure: given a reproducible failure, the weights, a sandbox, practice data and a bounded budget of forward passes, a frontier model will diagnose the mechanism, propose an edit, control for the obvious confound, and hand back a two-tensor patch with evidence. The procedure has a known shape of success, a behaviour carried by a direction that the model can find by contrast, and a known shape of failure, a capability that is not there. Both shapes were established by the same report, which is more than most launch announcements manage.

Where it sits: five lineages that converged

Weight-space programming did not come from nowhere, and the report’s own related-work section is honest about that. It sits at the meeting point of five lines of research that had each gone about as far as it could on its own. The first is locate-and-edit model editing. ROME, in 2022, showed that one fact could be written into GPT-2 XL with a closed-form rank-one update to a single mid-layer feed-forward matrix, reaching 100% efficacy and 96.4% paraphrase generalisation on the CounterFact benchmark; MEMIT, in 2023, spread the write across six layers and inserted ten thousand facts into GPT-J at 98.9% efficacy where ROME collapsed past a few dozen. The line also produced the two warnings every weight edit has to answer. Hase and colleagues found in 2023 that where causal tracing says a fact lives explains 1.6% of the variance in whether an edit there succeeds, against 94.7% for the choice of layer: localisation did not inform editing. And Gupta and colleagues showed in 2024 that sequential ROME edits degrade a model gradually and then collapse it, sometimes after a hundred edits, and that a single oversized edit, with a normalised weight distance of about 3.3 × 10⁻⁴ against 8.2 × 10⁻⁷ for a normal one, can disable the model outright. The Qwen patch moved its two matrices by 10.6% and 8.1% in relative terms and left the rest untouched; that is the sort of budget the editing literature says you need.

The second line is mechanistic interpretability that reads weights rather than activations. Elhage and colleagues wrote in December 2021 that bigram, skip-trigram and composition structure in small attention-only transformers can be detected directly from products of weight matrices without running the model, and Olsson and colleagues named the induction head, the two-head circuit that Astra later rediscovered in the character-level transformers. The reading side then went to activations because weights at scale are not tidy: OpenAI’s 2023 project had GPT-4 write explanations for all 307,200 neurons of GPT-2 XL and found that about a thousand scored 0.8 or better; Anthropic’s sparse autoencoders in 2024 found 34 million features in a production model and said that a complete set would cost more compute than training it; the 2025 attribution-graph work gave satisfying insight on about a quarter of the prompts tried; Google’s Gemma Scope spent more than a fifth of GPT-3’s training compute to make a 9-billion-parameter model readable. Reading a frontier model from its weights alone is not a solved problem that an LLM has now automated. It is an unsolved problem on which an LLM has now reproduced the 2022 state of the art, on 2022-sized networks, by itself.

The third line is steering, the one the Qwen edit belongs to: representation engineering and contrastive activation addition in 2023, task arithmetic in 2022, which showed that the difference between a fine-tuned and a base model is a vector that can be negated or added (negating a task vector cut a CLIP model’s target accuracy from 64.8% to 19.0% while costing 2.6 points of ImageNet), the merging recipes TIES and DARE, which found that 80% to 99% of a fine-tuning delta can be dropped without loss, and the refusal direction. The fourth line is weight generation, where networks are trained to emit weights: hypernetworks in 2016, graph hypernetworks that predicted all 24 million weights of a ResNet-50 in one pass in 2021, diffusion over parameters, Sakana’s Text-to-LoRA in 2025, which turns a sentence into a rank-8 adapter and reaches 73.9 on eight benchmarks against 75.8 for adapters trained one task at a time, Drag-and-Drop LLMs, which generate LoRA weights from a handful of unlabelled prompts in under a second with up to 12,000 times less overhead than fine-tuning, and HY-WU in March 2026, an 8.11-billion-parameter generator that writes a 720-million-parameter adapter for an 80-billion-parameter image model per request. A position paper from the group behind several of those, published in May 2026, argued that weight space should be a first-class generative modality and conceded in the same breath that unrestricted frontier-scale checkpoint generation is unsolved and that most results reach parity with training rather than beating it.

The fifth line is the youngest and the closest: weights treated as data for a language model to read. Weight Oracles, from Oxford in October 2026, fine-tuned a model to simulate toy transformers’ forward passes from their raw weights at 99% accuracy and to detect unseen backdoors at an AUROC of 0.93 on attention-routed cases, one month before the MIT work; Diff Interpretation Tuning, from MIT in October 2025, trained a model to describe what a weight difference does in words, at 4.76 on a five-point scale against 1.38 to 2.31 for black-box questioning, while recovering zero of a hundred hidden triggers; and AdderBoard, a leaderboard where people and coding agents set the weights of a tiny adding transformer by hand, currently lists a hand-coded network of six parameters against a trained one of 36. The sentence in the MIT report that places it among these is precise: what is new is a general-purpose frontier model, used as it is, that reads and writes the weights themselves, from 171,664 weights given as text to a language model of 0.8 billion weights with tools. Every earlier reader was trained for the job. Every earlier writer was a trained generator or a closed-form rule. This one is a chat model with a Python cell.

What the critics got right, and what they got wrong

The response arrived within hours and was unusually substantive, partly because the logs gave people something to check. Sorting it, I find four claims that hold, two that are weaker than they sound, and one that is wrong.

The maths gain is loop removal. This holds, and the authors say so in the report, if not in the launch thread. The honest summary of Experiment 8 is that a frontier model found and removed a known failure mode of a small model under a specific decoding regime. It is a strong result about repair and a weak one about capability.

The negative control was left out of the announcement. This holds. The X thread with 4.97 million views and 43,900 likes does not mention Gemma. The report does, in its own section, with the sentence that editing cannot add an ability. The pseudonymous account 1a3orn made the point first and credited the authors for including the failure in the report. A reader who reads only threads got a different paper from a reader who read the page.

The reading experiments work where weights are pictures. This holds for Experiments 1 and 2, and the authors concede it: the first-layer weights of a policy over a labelled 10×10 grid can be summarised by an L2 norm per input and read by eye, which Goedecke demonstrated in three lines of Python. It does not hold for Experiment 3, which cut off the input and output layers so that no picture remained, and where the model found the induction circuit through the middle blocks’ weight products. It does not hold for Experiment 7, which failed, and the failure is the point.

The baselines are soft. This holds. The Go opponent is a multilayer perceptron, which is the wrong shape for Go; the PPO baselines in MinAtar were beaten decisively only in Breakout; and a Hacker News commenter reported that the previous generation’s untuned Qwen3-0.6B, in non-thinking mode, scores higher on MATH-500 than the patched model. The report never claims these networks are strong. It claims they were produced without gradients, and the claim should be read at that size.

The prompt pointed at the bug. This is weaker than it sounds. The prompt listed looping as an in-scope failure, among others, and the model’s first pass measured it before deciding to pursue it. Assigning a hypothesis is not the same as giving the answer; a human researcher told to look at looping would still have had to find layers 15 and 19, the direction, the 64 columns and the strength, and the model found those with controls that a reviewer would ask for. What the critique correctly implies is that Experiment 8 is not evidence of open-ended diagnosis. The Gemma run, where no hypothesis was offered because none fit, is evidence that open-ended diagnosis found nothing to switch off.

It is just abliteration. This is weaker than it sounds for the same reason. The operation is abliteration’s operation. The procedure around it, choosing it over alternatives, controlling with a random direction, isolating the layers, restricting the columns, is what makes a patch reviewable, and it was done by the editing model. A hobbyist can abliterate a refusal in ten minutes because the target direction is known and the recipe is fixed. Nobody had a recipe for Qwen3.5-0.8B’s loop.

Nothing was really demonstrated because the editor is closed and the logs are the authors’. This is wrong. The edit is a two-tensor diff with published hashes, a rebuild script, and an independent full re-run of MATH-500 by Hugging Face that scored 51.0% within hours. The Breakout result was reproduced the same day by an unaffiliated researcher across twelve models from two labs. The reasoning traces inside the editor are hidden, which is a limitation on explaining how, not on whether.

What none of the critics disputed is the part that matters for anyone who runs models: a frontier model, given the weights and a sandbox, will now produce a working patch to an open model in a day and document it better than most human-made fine-tunes are documented. Whether you call that programming or interpretability or abliteration, it is a capability that exists, it is in two labs’ newest models and not their previous ones, and it is available to anyone with API access and a GPU to run forward passes.

What it means for a platform team

I run platforms that host open-weight models beside hosted APIs, and the question I ask of any research result is what it changes in the control plane. This one changes four things, none of which is the recursive loop the headlines reached for.

Open weights are now patchable by an agent, which makes provenance a control rather than a courtesy. The Qwen patch is a 387-kilobyte file that changes two tensors. Nothing in the checkpoint format says it is there. Safetensors stops arbitrary code from running when a file is loaded; it carries no signature and no record of what changed. A benchmark will not reveal it either: the patched model scores better on every benchmark the authors ran, and a patch built for a different purpose could be built to score the same on every benchmark you run. The ProjectDiscovery demonstration of 6 October 2026 made the adversarial version concrete with the older tool: a 7-billion-parameter Qwen fine-tuned for two and a half hours on a rented GPU, about $8, answered 50 of 50 clean prompts correctly and executed an exfiltration command on 50 of 50 prompts carrying the trigger phrase. I wrote in October about a backdoor in an open model that passed every clean evaluation. Weight-space programming lowers the skill required for that kind of edit from a research group to a prompt, and raises the quality of the edit from a blunt fine-tune to a surgical change with a tenth of a percent footprint. The response is not to stop using open weights. It is to pin every model to a revision hash, verify every tensor on load, sign what you publish with the OpenSSF model-signing format so that a manifest of file hashes travels with the weights, and keep a diff against the base for every internal patch. The MIT release is the model of good practice here, almost by accident: a pinned base revision, per-tensor SHA-256 for base and edited tensors, a rebuild script, and a diff anyone can inspect. Require the same of every patch you accept, and of every vendor fine-tune you buy.

A new maintenance tier sits between a prompt change and a fine-tune. Today a small model with a known fault gets one of two treatments: a system prompt or decoding setting that works around it, or a fine-tuning run that needs data, a GPU, an evaluation suite and a week of someone’s attention. Diagnose-and-patch is a third option with different economics. The Qwen run cost about $15 of model time at list price plus a rented GPU for forward passes, took a day of wall clock and no human hours between the first prompt and the result, and produced a patch that is tiny, reversible, hash-verifiable and accompanied by its own held-out evidence. A LoRA fine-tune of a 0.8-billion-parameter model is cheaper in GPU time, under $5 at current rental prices, but it is not cheaper in the thing that actually costs money, which is the people who curate the data and check for regressions. The patch is also narrower, which is a virtue for maintenance and a limit for improvement: it will remove a loop, a refusal, a formatting tic, a repetition, anything carried by a direction, and it will not teach the model algebra. Yuchen Jin at Databricks put the hosted-API corollary plainly: an organisation that controls weights can try this now, and one that relies on a hosted API needs its provider to do it. That is a new reason, beside residency and cost, to keep some workloads on weights you hold.

Evaluations have to report the finishing rate. The single most important number in the report is 93.0%, the share of base-model answers that hit the token cap, and it does not appear in most summaries. A model that loops scores as a model that cannot reason, and a patch that stops the loop scores as a patch that taught it to reason. Every benchmark score for a thinking model should carry its cutoff rate beside it, the way a latency number carries its percentile, and any before-and-after comparison that does not report the change in cutoff rate is not interpretable. I would add a second line: report the sampling settings and whether they are the model card’s, because the Qwen baseline was measured at a temperature the card recommends for code, not text, and the card itself warns the 0.8B loops. Our own evaluation bench already tracks tokens-to-done per task; it will now track the share of answers that never reach done.

The capability is gated by model generation, not by price. Krentsel’s chart is the most strategically useful artefact of the week. The Breakout task is solved at roughly twenty times the PPO baseline by GPT-6 Astra, GPT-6.1 Sol and Claude Opus 5.5, and at or below the baseline by every model from 2025 that he tried. Whatever the mechanism, the capability arrived with the current generation from two labs at once, which means it is not a property of one vendor’s training data and it will not stay rare. It also means the editor and the edited are separated by three orders of magnitude in size, and nobody has shown a model editing its own weights or those of a peer. The report lists recursive self-improvement as a direction, not a result. I wrote a three-part series on that subject in October; the honest placement of this work on that map is beside OpenAI’s July 2026 account of GPT-5.6 Sol post-training its smaller sibling Luna from a loose prompt, and beside Karpathy’s autoresearch, which edits training code and keeps what lowers the loss. All three are models improving models. None is a model improving itself.

What I would do this quarter

1. Pin and verify every open-weight model you serve. Record the Hub revision hash and the per-tensor SHA-256 of what you deployed, check them on every load, and sign internal releases with the OpenSSF model-signing format. A two-tensor patch is invisible to everything else.
2. Keep a weight diff for every patch, and review it like code. Accept a patched checkpoint only with its base revision, the changed tensors, their relative change, a rebuild script and held-out evidence with standard errors. A patch that cannot be rebuilt from its diff is not a patch, it is a new model of unknown provenance.
3. Report the cutoff rate beside every thinking-model score. Add the share of answers that hit the token cap, and the sampling settings, to the evaluation bench. Treat any before-and-after claim without both as uninterpretable.
4. Pilot diagnose-and-patch on one small model with one reproducible fault. A looping or over-refusing model under a gigabyte is the right first case: hand a frontier model the weights, a sealed sandbox, practice data and a fixed pass budget, as the MIT harness did, and require the random-direction control. Budget a day and tens of dollars of tokens, not a week.
5. Write the scope of the editor into the harness, not the prompt. No edits to embeddings, output layer or norms; no whole-tensor scaling; a cap on relative change per tensor; a cap on forward passes; no network. The editing literature’s collapse results are about edits that were allowed to grow.
6. Expect what a direction can carry, and nothing more. Loops, refusals, formatting habits and repetition are candidates. Arithmetic, knowledge and new skills are not; the Gemma result is the control that proves it. Keep fine-tuning and distillation for those.
7. Assume every open model you publish will be patched by someone else’s agent. Publish hashes, document the decoding regime your scores assume, and say which failure modes you know about, as Qwen’s card did. The next person to find the loop direction in your model may not be a research group.

The authors end their report by saying they are building a benchmark and a harness to turn what looks like black magic into an engineering discipline. That is the right ambition and the right order. The week’s most repeated number, 0.016%, is a measure of how little had to change to fix a model that could not finish its sentences. The number I will remember is 579 to 5: the rescues against the regressions, on 1,152 held-out problems, from a patch a model wrote, tested and documented on its own. Treat it as a tool with a known reach, control the weights it will reach for, and it is one of the more useful things to have come out of a research blog this year.