For years, one of the most unsettling facts about artificial intelligence has been how difficult it is to explain what happens inside a trained neural network. Engineers can measure a model’s performance, inspect its weights, and test its outputs, yet still struggle to give a convincing account of why a particular internal component responds as it does. A new essay by researcher Nick Cammarata argues that this difficulty may say as much about our investigative tools as it does about the networks themselves.
Published on October 15, 2026, “Neural Networks Are Already Clean” has quickly attracted attention in AI research circles. Cammarata, now at Goodfire and previously involved in foundational interpretability work at OpenAI, revisits examples that helped persuade researchers that neural networks encode concepts in hopelessly tangled ways. He suggests that many apparent tangles are actually understandable forms of generalization, provided we study them with better methods.
His most attention-grabbing preliminary estimate concerns Qwen3-32B: roughly 90% of the neuron poles examined appear to have one principal function, often with related extensions. That does not demonstrate that 90% of everything a language model does has been understood. The difference between understanding many individual components and explaining an entire intelligent system is central to evaluating this claim.
What does it mean to understand an AI neuron?
Artificial neurons are computational units whose activations change as data moves through a model. The biological metaphor can be useful, but an artificial neuron is not a miniature person holding an idea. Its activity is a numerical signal, affected by the input, surrounding units, and learned parameters. To understand that neuron, a researcher needs more than an appealing label. A useful explanation predicts when it activates, what makes it less active, how it interacts with neighboring components, and what changes when the researcher intervenes.
This is the ambition of mechanistic interpretability: discovering internal structures and causal processes rather than inferring the model’s behavior solely from its responses. It is somewhat like inspecting a computer program at the level of its working mechanisms, except that the program was learned through optimization rather than written as explicit, human-readable instructions.
Historically, researchers encountered neurons that responded to apparently unrelated things. A famous unit in an early image-recognition model activated for cat faces and the fronts of cars. If a neuron appeared to detect both categories, perhaps the network was combining unrelated concepts inside the same limited representational space.
The cat-and-car neuron: a different explanation
Cammarata returns to that example with a simple question: what do the triggering images actually have in common? His proposed answer is a pointed, V-like shape, visible in certain animal faces, vehicle fronts, and other objects. He reports examining the neuron’s upstream inputs, downstream effects, and the patterns that push its activation in the opposite direction. In that account, the common structure is not an accidental collision between cats and cars. It is the shared visual feature that the neuron has learned to use.
Why did this not seem obvious before? A model visualization does not simply reveal the neuron’s meaning. It generates or selects images designed to increase activation, and that procedure can emphasize misleading details. A picture full of fur or chrome may make the detector look like a confused collection of object categories when the relevant signal is a more abstract geometric relationship.
The episode is a useful lesson about measurement. Scientific instruments influence which patterns become visible. When a microscope produces an unclear image, one possibility is that the object is inherently chaotic; another is that the microscope is poorly calibrated. Cammarata argues that a substantial part of early interpretability pessimism followed from mistaking the latter for the former.
Superposition versus recursive generalization
The broader debate concerns superposition. In an influential picture, a network may represent more distinct features than it has obvious neuron-sized slots, overlapping these features across dimensions. This is a meaningful theoretical possibility and has shaped research into techniques that recover interpretable features from distributed activations, particularly sparse autoencoders.
Cammarata does not merely say that researchers used a few bad visualizations. He proposes a different dominant explanation for much of what looked like arbitrary feature packing: recursive generalization. A neuron may begin with a narrow pattern, then expand toward nearby or conceptually related patterns as training progresses. A broader category can develop increasingly specific branches, much as related ideas are organized into a hierarchy without being completely independent.
These stories make different predictions. Arbitrary packing suggests that apparently unrelated meanings may share a unit without a useful conceptual relationship. Recursive generalization suggests that the relationship itself is informative, even when a human observer needs time to discover it. Under the second picture, the native neurons and their connections become suitable objects of direct investigation.
It would be premature, however, to conclude that superposition never occurs. Network architectures vary, mathematical arguments for overlapping representations remain relevant, and interpretability depends on what counts as a feature and which components are being measured. Cammarata’s essay is explicitly framed as a working hypothesis, not a completed proof that one account applies universally.
Where does the 90% figure come from?
The essay reports preliminary audits of Qwen3-32B in which approximately 90% of the examined neuron poles were assigned one main job, about 7% had two related jobs, and roughly 2–4% remained unexplained. “Neuron poles” is a more specific object of analysis than a blanket claim about every neuron, attention head, residual-stream direction, or circuit in the whole model. The percentages also depend on how the analysis groups related responses and how much weight it assigns weak activations.
There are additional distinctions that matter. A unit may be interpretable under a helpful explanation without that explanation identifying all of its causal effects. It may behave intelligibly on the inspected examples while showing other roles on an unfamiliar distribution. And a collection of readable parts can still form a system whose joint behavior is difficult to anticipate. In software engineering, understanding what individual functions do does not automatically make the behavior of an entire distributed system transparent.
Cammarata compares this preliminary picture with earlier automated interpretability efforts. The OpenAI project he helped start in 2023 found only about 1.7% of analyzed neurons interpretable under its generous metric. But the old and new figures do not constitute a controlled improvement on the same task. The models, explanation methods, evaluation criteria, and investigative tools differ. The contrast highlights a potentially important methodological shift, not a simple benchmark in which one technique definitively wins.
Why are larger networks sometimes easier to inspect?
One surprising aspect of the proposal is that more complex networks need not always be more confusing locally. Cammarata describes experiments with modernized vision models in which late-layer neurons become much more legible than they were in the older InceptionV1 architecture, even at comparable sizes and with comparable training data. He reports that 99–100% of the last-layer neurons in a particular modified model are readily interpretable under his approach.
That finding is narrower than the language-model claim and should stay separate from it. Still, the idea has an appealing logic: when a model has enough representational capacity, individual units may specialize in cleaner, more particular distinctions. A smaller network may force a unit to cover a broad family of related features. The larger system can distribute that family across more specialized units, making each one easier to name and test.
Understanding could also compound. If investigators map a few mechanisms, those explanations can constrain the interpretation of their neighbors. Cammarata likens the process to solving a crossword puzzle, where answers in one direction provide evidence about another. Modern AI agents might help scale this work by proposing hypotheses, running targeted experiments, and building navigable maps of circuits.
What remains unproven?
The essay is ambitious, but much of the most consequential language-model evidence remains preliminary. Cammarata says a technical companion paper will provide additional detail. Independently reproducing the findings across architectures, tasks, and evaluation methods will matter more than the amount of attention the essay receives. The best test of an interpretation is not how intuitively persuasive it sounds, but whether it survives interventions designed to make it fail.
Several open questions deserve particular attention. How consistently can independent researchers assign the same function to a neuron? Can its behavior be predicted on new inputs rather than described retrospectively? Does changing that neuron produce the expected downstream effect? How much of the model’s important behavior depends on interactions that are not visible from local neuron explanations? And can the same methods illuminate safety-relevant phenomena such as deception or strategic planning?
Those questions explain why interpreting a neuron’s behavior is not equivalent to guaranteeing that a model is safe, truthful, or controllable. A medical analogy is useful here: identifying the functions of many biological components does not mean predicting every consequence of a complex interaction among them. Better anatomical knowledge is valuable precisely because it creates a foundation for more rigorous causal understanding, not because it abolishes uncertainty.
Could this change AI safety and scientific research?
If the hypothesis withstands scrutiny, its greatest consequence may be methodological. Instead of first constructing an elaborate alternate representation of a network, researchers could investigate more of the original system directly. They could ask which units matter for a behavior, test the predicted effects of modifying them, and connect local mechanisms into larger explanations. That would give interpretability research a more experimental, iterative character.
Even an incomplete version of that progress would be valuable. A clearer map of neural mechanisms might improve debugging, targeted evaluation, detection of unexpected model behavior, and our ability to compare systems with different training histories. These applications would require distinct validation, however; none follows automatically from a high percentage of individually legible neuron poles.
There is a connection here to another recent InsightArea discussion of whether AI can help verify scientific papers. Both stories concern the growing separation between producing impressive results and explaining why those results should be trusted. In one case, researchers want to check complex arguments. In the other, they want to inspect the machinery producing increasingly sophisticated behavior.
The possibility that neural networks are less mysterious than we assumed is exciting precisely because it can be investigated. Cammarata has offered concrete examples, tools, and a bold hypothesis. The next step is for the wider research community to try to disprove, refine, or extend it. If the work survives that process, the phrase “AI black box” may become a less useful description of what modern science can actually understand. Until then, it remains a hypothesis worth testing rather than a mystery already solved.
By Costin Liculescu, creator of InsightArea, exploring the connections among artificial intelligence, computer science, mathematics, and scientific reasoning.
Sources and further reading: Nick Cammarata, Neural Networks Are Already Clean (Goodfire, October 15, 2026); Goodfire Research; Olah and colleagues, Zoom In: An Introduction to Circuits (Distill, 2020); Anthropic, How employment is changing at Anthropic (October 16, 2026).
Comments are closed.