Researchers at MIT CSAIL said Monday that a frontier language model, GPT-6 Astra, can inspect and rewrite neural network weights directly, without gradient descent or a training loop¹.

In a blog post dated Nov. 2, 2026, Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab call the approach Weight-Space Programming: an LLM reads a network’s weights around a natural-language specification and writes new values¹.

On Qwen3.5-0.8B, they say Astra found a weight-space issue tied to reasoning loops that Qwen’s model card warns about, then edited 0.016% of the weights¹. With the same sampling settings, they report MATH-500 rose from 7.9% to 48.1% and IFEval roughly tripled¹.

The authors also describe smaller tests¹. They say Astra, given raw weights as text and no game name, identified which Atari-style game PPO networks were trained on¹. On a failing car-control policy, they say a single Astra edit of 3.5% of the weights made the car succeed in all 1,024 evaluation episodes¹.

Asked to write MinAtar game networks from scratch with no rollouts, data or gradients, they say Astra’s networks beat PPO trained for 100 million steps on Breakout, Seaquest and Asterix¹. Against PPO trained for 500 million steps, they report Astra’s Breakout network scored about 20 times as much¹.

They write that they did not let Astra write training loops, and that it instead reasoned about what parts of a network compute¹. They released logs at github.com/RikiyaT/lm_understand-logs and said they are building a benchmark and harness for the method¹.

The numbers and interpretations are the authors’ own; the post is not a peer-reviewed paper, and no independent replication is cited¹.

Sources

1. rikiyat.github.ioSparks of Weight-Space Programming: LLMs can natively read & write neural network weights1:21 PM

Read the original at rikiyat.github.io ↗