Claude Opus Wrote a Neural Network by Hand. It Used 166 Weights.

A new neural network scored 100 per cent accuracy on a hidden test set across 13 algorithm tasks and 26 independent runs. It was not trained. No backpropagation, no gradient descent. Claude Opus 5.5 wrote it, one weight at a time, in code.

Diagram of the interpretable weight space programming method
The iWSP method: an LLM writes every weight as code, with no training allowed. (Source: research paper via 36Kr)

Skip the optimiser

Researchers from Microsoft Research, Johns Hopkins University, Columbia University and New York University published a method they call interpretable weight-space programming, or iWSP. A frontier language model can bypass the traditional optimiser, read a fixed evaluation score and write the algorithm directly into a Transformer’s weights using Python code.

Diagram from an AI interpretability paper
Figures from the interpretable weight-space programming paper. (Source: research paper via 36Kr)

Deep learning is often described as alchemy. Engineers turn thousands of knobs with no view inside the machine, judging success only by the error signal. Which knob controls logic, which controls memory, nobody can say. That is tolerable in engineering when prediction is accurate. In neuroscience or physics, where explanation is the goal, it is fatal. A deep encoding model may predict brain responses precisely and still say nothing about which features drove them.

Under iWSP, Opus is not allowed any gradient-based training. On each iteration it sees a single validation score. It must draw on prior knowledge, design a computational logic and then explicitly write the features and the numbers into the weights. It evaluates its own creation and revises the code against that one score. What emerges is a Transformer a person can read from start to finish, because every line was deliberately written.

Smarter models wrote smaller networks

The striking result is not only that an AI built an AI. It is that stronger models produced smaller ones. Haiku 5.5 wrote networks with a median of 100,000 parameters. Sonnet 5.5 needed 41,600. Opus 5.5 wrote networks with a median of 166 parameters. In an era measured in hundreds of billions of parameters, 166 is a speck of dust.

The logs explain why. Opus first wrote a network of roughly 3,400 parameters to get the logic working, then trimmed it repeatedly once it reached full marks, stripping redundant computation and compressing matrices until about 200 remained. The analogy in the paper is a student who has memorised every case versus one who understands the formula. If the correct logic is found, scale is not required.

A microscope for science, with limits

The team applied iWSP to brain-encoding models. In a language fMRI experiment, a conventionally trained black box exploited a confound in the dataset to score well. The transparent model written by the AI exposed it immediately. In visual-cortex tasks it outperformed existing interpretable models and showed how early and higher visual areas compute differently. On auditory electrocorticography data it identified interpretable acoustic features that predict neural activity.

The method hits a wall in natural language. Emotion classification is full of irony, ambiguity and multiple meanings, with no clean formula to write. Opus inflated its model to about 190,000 parameters and still reached only 72 per cent. On a paraphrase task it failed to beat the baseline of guessing the most common answer. iWSP is good at recombining known algorithms and computational modules. Emergent representation for complex language still requires data and gradient descent.

That boundary matters. This is not proof that foundation models no longer need training. It is proof that in a specific setting a neural network can escape blind evolution, and that the resulting logic can be read rather than guessed.

Editor’s note: This is an adapted translation of the original 36Kr report. It has been trimmed and restructured for readability for an international business audience. The full original (in Chinese) is at https://m.36kr.com/p/4075960710566792.

Leave a comment