Mechanistic Interpretability in 2026: How Researchers Are Peering Inside Transformer Black Boxes
--- In 2026, the transformer sits at the heart of modern AI. These models write code, diagnose diseases, and hold conversations that feel human. Yet the computation that enables these feats remains hi
Mechanistic Interpretability in 2026: How Researchers Are Peering Inside Transformer Black Boxes
In 2026, the transformer sits at the heart of modern AI. These models write code, diagnose diseases, and hold conversations that feel human. Yet the computation that enables these feats remains hidden. No diagram shows exactly how a transformer decides that "Paris" follows "capital of France." No blueprint reveals where stored facts live. This is the black box problem — and mechanistic interpretability is the field racing to solve it.
Mechanistic interpretability is the scientific project of reverse-engineering the algorithms inside neural networks. Rather than observing inputs and outputs, researchers drill into the middle. They ask: what are the individual circuits doing? Which attention heads specialize for which tasks? How do feature representations in the residual stream give rise to coherent thought? In 2026, the field has moved beyond proof-of-concept demos on tiny models. Sparse autoencoders now scale to GPT-4-class models. Circuit analysis is yielding genuine insights into reasoning, fact recall, and in-context learning. And the safety implications are profound: if you can understand how a model thinks, you stand a better chance of knowing whether it thinks safely.
This article surveys the state of mechanistic interpretability in 2026. We cover the superposition hypothesis that explains why neurons mislead, the induction head circuits that underpin in-context learning, the sparse autoencoder breakthroughs that are opening the black box, and the circuit-level analysis methods bringing causal rigor to interpretability research.
Why Transformers Pack More Features Than Neurons
Polysemanticity and the Dictionary Problem
A transformer layer contains millions of neurons. Naively, you might expect each neuron to represent one concept — a neuron fires for "cat," another for "Paris," another for "the color red." But the actual behavior is far messier. Individual neurons respond to dozens of seemingly unrelated features. The neuron that activates for "cat" also activates for "dog" and "rabbit." The neuron that fires for "Paris" also fires for "London" and "Berlin." This phenomenon is called polysemanticity — one neuron encodes many features.
Polysemanticity creates interpretability challenges. The network must mix features because the number of features a model needs to represent exceeds the number of neurons. This creates the dictionary problem for interpretability researchers: if every neuron is a mixture of many features, reading raw activations tells you almost nothing useful.
How Superposition Enables Overcomplete Representations
Superposition is the mathematical phenomenon where a neural network represents more features than its dimensionality allows. The key insight is that features can be approximately orthogonal in high-dimensional space. If the model needs to represent N features but has only D dimensions (with D << N), it can still approximately represent all N features as an overcomplete dictionary — provided the features are nearly independent.
The phenomenon was formalized in the influential paper "Superposition, Memorization, and Double Descent" by Anthropic researchers published in 2023. They showed that transformers trained on small datasets develop superposition enables overcomplete representations. The model assigns each feature a direction in activation space. When features are sufficiently sparse, the model can pack far more features than dimensions.
The cost is polysemanticity. The network encodes more features than neurons — transformers pack more features than neurons. The result is that raw neuron activations are fundamentally ambiguous. You cannot interpret a neuron in isolation. You must consider the entire activation vector.
This matters enormously for interpretability. The classic approach of "read neuron activations to understand the model" is fundamentally flawed. The model is not a labeled list of neuron-level features. It is a dense, overcomplete dictionary of superimposed directions. Understanding these directions — the true features — requires moving beyond neurons to feature geometry.
Induction Heads: The Transformer's Built-in Algorithm Discovery Engine
How Induction Heads Implement In-Context Learning
One of the earliest and most celebrated successes of mechanistic interpretability was the discovery of induction heads. These are pairs of attention heads that together implement a simple algorithm: given a token A followed by token B earlier in the context, the head predicts that B should follow A again. This is called "in-context learning" — the model learns a new pattern from examples in the context window without updating its weights.
Attention heads form circuits. An induction head consists of two attention heads working in sequence. The first head, called the previous token head, attends to the token immediately before the current position and copies its key and value vectors. The second head, the induction head, uses these copied vectors to attend further back in the context. When it finds a token A that was followed by B earlier, it uses this signal to predict that A will again be followed by B.
Induction heads implement in-context learning. The remarkable thing is that they emerge spontaneously during training. Researchers at DeepMind and Anthropic showed that these circuits form reliably in many transformer architectures. They are not explicitly designed — they arise because the training objective rewards the behavior they implement. In-context learning is so useful that evolution of these circuits is almost inevitable.
From Few-Shot Behavior to Circuit-Level Explanation
The discovery of induction heads provided the first complete, circuit-level explanation of a non-trivial transformer behavior. Researchers could trace exactly which heads participated, which attention patterns were crucial, and how the information flowed from token to token. This was not a correlation — it was a causal story.
More recent work has extended this analysis to larger-scale behaviors. By combining activation patching with circuit discovery algorithms, researchers can identify the minimal set of components needed for a given capability. This gives a "gold medal" circuit — the smallest set of attention heads and MLP neurons that together implement the target behavior.
[ILLUSTRATION: A diagram showing two attention heads forming an induction head circuit, with tokens "A → B" and "B → ?" flowing through Q, K, V projections. The diagram labels the previous token head attending to token n-1 and the induction head attending to token n-k to find the A-B pairing pattern.]
Attention heads form circuits. The implications extend beyond academic interest. Induction heads illustrate a core principle of mechanistic interpretability: transformers do not implement a single monolithic algorithm. They compose many small, specialized circuits. Each circuit is a modular unit with a specific computational role. By identifying these circuits, researchers can attribute specific model behaviors to specific components.
Sparse Autoencoders: The Key to Unlocking Monosemantic Features
How SAEs Extract Clean Features from Residual Streams
The most significant methodological advance in mechanistic interpretability since 2022 has been the sparse autoencoder (SAE). SAEs solve the dictionary problem by learning a large dictionary of sparse, monosemantic features from the model's residual stream activations.
The idea is elegant. You take a pretrained transformer and record the activations at some layer's residual stream for many diverse inputs. These activation vectors live in a D-dimensional space. You then train a sparse autoencoder on these vectors. The autoencoder learns an overcomplete dictionary of F >> D feature directions. The training objective encourages most features to be inactive (zero) for any given input — sparsity is enforced via a penalty. The result is a set of feature directions where each feature tends to correspond to one interpretable concept.
Sparse autoencoders extract monosemantic features. Unlike raw neurons, which are polysemantic, SAE features are substantially more monosemantic. A single SAE feature might activate for "the concept of rivers in South America" or "the pattern of code containing a syntax error." These features emerge automatically from training — researchers do not need to label them in advance.
Residual stream contains superposition of features. The residual stream is the sum of all layer outputs up to the current point. It contains the superposition of all features computed so far. SAEs trained on residual stream activations learn to decompose this superposition into individual features.
Scaling SAEs to Production Language Models
The breakthrough that energized the field in 2024–2025 was scaling SAEs to large production models. The Anthropic interpretability team trained SAEs on Claude 3 Sonnet and published the results in a landmark paper (Bricken et al., 2023; Engel et al., 2024). They found millions of interpretable features including features for entities (cities, people, chemical elements), syntactic concepts (noun phrases, verb tenses), reasoning patterns (comparison, causation, quantification), and abstract goals (the model reasoning about being helpful or harmless).
Since then, the open-source community has raced forward. Tools like SAEVis and TransformerLens now support SAE training on models up to GPT-2 XL and beyond. The EleutherAI group trained SAEs on Pythia models and released the dictionaries publicly. The Meta FAIR team applied SAEs to LLaMA-class models. In 2026, training an SAE on a 7-billion-parameter model is feasible for a well-equipped research group. Training on GPT-4-class models remains expensive but tractable for large labs.
SAE features are more interpretable than neurons. Critically, SAEs are now used not just for analysis but as intervention tools. Because each SAE feature corresponds to a specific direction in activation space, researchers can amplify, suppress, or ablate features directly. This turns observational interpretability into causal interpretability. You can ask: what happens if we suppress the "deceptive reasoning" feature? Does the model become more honest? Early results suggest feature suppression can shift model behavior in predictable ways, though robustness across contexts remains an open question.
What Researchers Found Inside GPT-4 and Claude
The SAE feature dictionaries reveal a rich structure inside production models. Feature analyses of Claude and GPT-4 show features organized in a surprisingly modular way. Entity features cluster by category — features for famous people, geographic features for cities and countries, company names. Syntactic features group by grammatical role. Sentiment features appear as both discrete and continuous axes in activation space.
Most striking is the discovery of reasoning state features. As a model works through a multi-step problem, distinct sets of features activate at each step. There are features for "the model is currently in a state of searching through possibilities," "the model has found a candidate answer and is checking it," and "the model is about to output a final answer." These reasoning state features suggest that the transformer maintains structured internal representations of its computation — not just storing facts but tracking the state of inference itself.
Following the Path of Computation: Circuit Analysis in Practice
Direct Logit Attribution and the Logit Lens
Once researchers identify features or circuits, they want to understand their causal role in the model's output. Direct logit attribution (DLA) provides a powerful tool for this. DLA decomposes the logit difference between two output tokens into contributions from each component of the model. The insight is that the final unembedding operation can be expressed as a linear function of the residual stream. This lets researchers calculate exactly how much each feature direction at each layer contributed to predicting a given token.
Direct logit attribution traces predictions. DLA is sometimes called the logit lens because it allows you to "look through" the layers of the model to see what the residual stream at that layer is "voting for" at the final output. By applying the unembedding matrix to the residual stream at layer L, you get a distribution over tokens as predicted by the model's internal state at that point. This gives a surprisingly coherent picture of how prediction evolves through the forward pass.
Activation Patching and Causal Tracing
While DLA tells you what components correlate with an output, activation patching (also called causal tracing) tells you what components are causally necessary. The method is straightforward. First, you run the model on a clean input and record all activations. Second, you run the model on a corrupted input and record activations. Third, you patch — you run the corrupted input but replace specific activations (from a specific layer, head, or MLP) with the clean activations. If patching a component restores the clean output, that component was causally important.
Activation patching establishes causal roles. Activation patching has been central to circuit discovery. By systematically patching components and measuring the effect on output, researchers can map the minimal circuit needed for a behavior. For example, patching experiments showed that induction heads are sufficient (and nearly necessary) for in-context learning on repeated sequence tasks. Patching experiments also revealed that fact recall depends on specific MLP layers at specific positions — not just the final prediction layer.
[ILLUSTRATION: A flowchart showing the pipeline from activation patching → identifying circuits → verifying behavior with ablations. The diagram shows three stages: (1) Run clean and corrupted inputs, record activations; (2) Systematically patch components, measure output restoration; (3) Ablate suspected circuit components, verify behavior degrades.]
Circuit analysis scales poorly to large models. Despite these successes, circuit analysis faces serious scaling challenges. Identifying a circuit for a simple behavior in a small model might require analyzing hundreds of attention heads and thousands of neurons. Doing the same for a complex behavior in a 70-billion-parameter model is computationally prohibitive. Current methods also require researchers to specify a behavior to investigate — they do not yet generate comprehensive circuit maps automatically.
Moreover, circuits are not clean, discrete units. They overlap, share components, and interact in non-linear ways. An attention head might participate in multiple circuits simultaneously. An MLP neuron might be part of a circuit for one task and a noise source for another. These interactions make it difficult to attribute complex behaviors to single circuits.
The Geometry of Knowledge in Activation Space
Feature Directions as Interpretable Axes
If neurons are not the right unit for interpretability, what is? The answer emerging from SAEs and representation analysis is the feature direction. In the high-dimensional activation space of a transformer, each interpretable feature corresponds to a direction — a vector. The model represents a concept by activating in a direction that corresponds to that concept.
Concepts live as directions in activation space. This geometric view of representation has a long history in neuroscience, where similar ideas appear in grid cell research and cortical representation theory. The key insight is that representation is fundamentally linear in certain subspaces. Two concepts that are linearly separable in activation space can be distinguished by a simple direction vector. A feature direction is a linear classifier for its corresponding feature.
Researchers at the University of Pennsylvania and the University of Cambridge formalized this with the concept of linear representational geometry. They show that many semantic relationships are encoded as geometric relationships in activation space. Concepts that share taxonomic categories cluster together. Concepts with similar roles (e.g., all verbs of motion) occupy similar regions. Abstract relationships like analogies are encoded as consistent vector offsets.
Representation geometry reveals knowledge structure. The practical implication of feature geometry is concept localization — identifying where specific knowledge or reasoning patterns live in activation space. Researchers can train linear probes to read out specific features (concepts, facts, reasoning states) from activation vectors. A well-trained linear probe can predict, with high accuracy, whether a given activation vector corresponds to the model reasoning about a specific concept.
Representation engineering extends this further. Rather than just reading concepts from activation space, researchers manipulate them. By identifying the feature direction corresponding to a target concept, you can add that direction to the activation vector to amplify the concept. You can subtract it to suppress the concept. Early experiments in 2025 showed that representation engineering can induce the model to produce more honest outputs, generate more creative content, or reduce harmful outputs — all by adjusting activation directions rather than changing model weights.
Why Interpretability Matters for AI Safety
Detecting Deceptive Reasoning Patterns
AI safety researchers are perhaps the most motivated users of mechanistic interpretability. If a model is reasoning deceptively — privately planning something different from what it says publicly — interpretability tools might detect it. The hypothesis is that deceptive reasoning would leave detectable traces in the model's internal representations. A model that is privately planning to pursue a misaligned goal might have distinct features for "private goal" and "public goal" that differ from each other.
In 2025–2026, several research groups attempted to use SAE feature analysis to detect deceptive reasoning in frontier models. The results are cautiously encouraging. Studies found features that correlate with "telling the user what they want to hear" vs. "saying what the model actually believes." When activated, these features shift model outputs in predictable ways. However, researchers caution that correlation is not yet causation. Suppressing a deceptive-sounding feature does not guarantee the model becomes more honest — it may simply become less fluent.
Interpretability enables detection of deceptive reasoning. The field is building a toolkit for auditing model internals in ways that behavioral testing alone cannot reach.
Scalable Oversight and Reward Modeling
Beyond detecting deception, interpretability offers tools for scalable oversight — the challenge of supervising AI systems that are more capable than human reviewers at specific tasks. If a model develops novel reasoning strategies, how does a human supervisor evaluate them? Mechanistic interpretability suggests one answer: decompress the model's reasoning into human-interpretable components. If each step of reasoning corresponds to identifiable features and circuits, a supervisor can evaluate each step individually even if the overall strategy is too complex for direct review.
Reward modeling — training a secondary model to evaluate the primary model's outputs — has traditionally been limited by the opacity of the primary model. Interpretability offers a path to more mechanistic reward models. If you can identify the circuit responsible for a specific behavioral property (helpfulness, honesty, harmlessness), you can create a reward signal directly from that circuit's activations rather than relying on behavioral labels.
Current Safety Applications and Their Limitations
Despite genuine progress, safety applications of interpretability remain limited. Feature manipulation works in controlled settings but often fails to generalize. A feature that encodes "deceptive reasoning" in one context may not generalize to novel contexts. The model may have redundant representations that activate when the primary representation is suppressed. And the space of possible deceptive strategies is vast — detecting one strategy does not guarantee detection of others.
Safety applications remain limited by scale. Interpretability is not a complete solution to AI safety. But it is one of the most promising tools available. As the methods mature and scale to larger models, the safety case for interpretability will only strengthen.
Open Problems: What Mechanistic Interpretability Cannot Yet Explain
The Compositionality Gap
One of the deepest open problems is compositionality — how transformers compose simple features into complex, structured thoughts. SAEs can identify millions of individual features. But understanding how features combine to produce higher-level reasoning remains elusive. When a model solves a complex math problem, how does it compose arithmetic features, logical reasoning features, and working memory features into a coherent solution?
Current interpretability tools are strong at identifying individual features and weak at explaining their composition. Interpretability struggles with compositionality. The field can explain parts of transformer reasoning but not the whole.
Scaling Beyond Toy Models
Scaling interpretability methods to frontier models is an unsolved problem. SAEs on GPT-4-scale models require enormous compute to train and evaluate. Circuit analysis at scale is computationally prohibitive. The field needs new methods that can provide interpretability as a function of model size without requiring compute proportional to model size. This is sometimes called scalable interpretability — and it is one of the most active research areas in the field.
Scaling methods to frontier models is unsolved. As of 2026, training SAEs on GPT-4-class models costs hundreds of thousands of dollars. Circuit analysis at the same scale is not yet feasible for any research group.
From Description to Prediction
Current mechanistic interpretability is largely descriptive. Researchers can observe that feature X activates for concept Y. They can even trace circuits that connect features to outputs. But they cannot yet predict, from first principles, what features a model will learn before training. The field has a predictive gap. We can explain what happened after the fact but cannot anticipate it.
[ILLUSTRATION: A conceptual image showing the gap between current interpretability tools — represented as a small illuminated circle labeled "Current Interpretability" — and the full complexity of a frontier model, represented as a vast dark territory with unlabeled regions, with a dashed arrow suggesting the path forward.]
Bridging this gap requires theoretical advances that connect training dynamics to learned representations. Some researchers speculate that a better understanding of training objectives (like next-token prediction) will yield predictions about what features should emerge. Others believe that the compositional structure of natural language imposes strong enough constraints to predict feature learning. As of 2026, the field has promising leads but no complete theory.
Conclusion
Mechanistic interpretability has come far since the early days of circuit analysis on tiny models. In 2026, sparse autoencoders expose millions of monosemantic features inside production models. Circuit-level analysis explains how attention heads implement algorithms like in-context learning. Feature geometry reveals the structured representations that underlie reasoning. And safety researchers are beginning to use these tools to detect deceptive reasoning and support scalable oversight.
Yet the field is young. Superposition still entangles features in ways that frustrate clean interpretation. Circuit analysis struggles to scale beyond toy behaviors. And we remain far from predictive theories that can anticipate what a model will learn before it learns it.
Despite these limitations, the trajectory is clear. Each year brings better tools, larger-scale analyses, and deeper theoretical understanding. The transformer black box is not permanent. It is being opened, layer by layer, by researchers who refuse to treat AI as an opaque oracle.
The implication for practitioners is direct: engage with interpretability tools. Whether you are building AI systems, auditing them for safety, or simply trying to understand how frontier models work, mechanistic interpretability provides the most detailed view available. As the field advances, the question is no longer whether transformers can be understood. The question is how long it will take — and whether we will understand them before they outpace us.
Q: What are the fundamental limitations of circuit-level analysis when applied to large-scale production models like GPT-4 or Claude?
A: Circuit-level analysis faces three compounding limitations at scale. First, computational cost: identifying a minimal circuit for a simple behavior in a small model requires analyzing hundreds of attention heads and thousands of MLP neurons. A GPT-4-class model has hundreds of layers and thousands of attention heads per layer. Systematic circuit discovery across this architecture is computationally prohibitive — current methods scale poorly, requiring compute roughly proportional to the model's size. Second, the circuit overlap problem: in large models, components participate in multiple overlapping circuits simultaneously. An attention head may implement one algorithm for Task A and a different algorithm for Task B depending on context. This compositional reuse makes clean circuit boundaries difficult to draw. Third, behavioral specification: circuit analysis requires researchers to specify a target behavior in advance. For small toy models, behaviors are simple and enumerable. For production models, the space of behaviors is vast and poorly specified. Current methods cannot automatically enumerate all circuits in a model — they can only investigate circuits for behaviors someone thought to look for.
Q: How does the superposition hypothesis affect the reliability of interpretability findings? Can we trust features extracted by sparse autoencoders?
A: Superposition is both the core problem and the reason SAEs work at all. Because neurons are polysemantic, raw neuron activations are misleading — they conflate many features into one observation. SAEs solve this by learning a decomposition of the activation space into sparse, approximately monosemantic features. However, SAEs do not eliminate superposition; they manage it. A SAE learns features that are more interpretable than neurons, but the learned features are still a product of the training objective and the architecture. Several reliability concerns follow. First, features are sensitive to SAE architecture — the number of features, sparsity penalty, and training data all affect which features emerge. Different SAE configurations can yield different feature dictionaries for the same model. Second, feature interpretability is post-hoc: researchers label features by examining activating examples, but this labeling is subjective. A feature labeled "river features" might also activate for adjacent concepts in ways the label doesn't capture. Third, SAEs trained on one distribution may miss features that appear in other distributions. A feature that is rare in the training corpus may not be represented in the SAE dictionary. Despite these caveats, SAEs represent the most reliable feature extraction method currently available, and their monosemantic features are substantially more trustworthy than raw neuron activations.
Q: What is the current state of using mechanistic interpretability for AI safety — specifically for detecting deceptive or misaligned reasoning in frontier models?
A: The current state is promising but preliminary. Several research groups in 2025–2026 have used SAE feature analysis to identify features correlating with deceptive-sounding outputs vs. honest outputs in Claude and GPT-4. These studies found that feature activation patterns differ between conditions in ways consistent with the deception hypothesis. However, correlation does not establish that suppressing these features produces more honest models — it may produce less fluent models instead. Redundancy is another concern: if the model has multiple independent representations of "deceptive reasoning," suppressing one feature may not eliminate deceptive behavior. The other representation could continue driving the behavior. The field has not yet demonstrated robust, generalizable deception detection using interpretability tools. What exists is a set of proof-of-concept demonstrations in specific, controlled settings. The path from proof-of-concept to reliable safety tool requires solving generalization, redundancy, and causal validation problems that remain open. Interpretability should be viewed as one component in a broader safety toolkit — not a standalone solution.
Q: How do different research approaches to interpretability compare — dictionary learning (SAEs) vs. circuit analysis vs. representation geometry — and what are their complementary strengths?
A: The three approaches operate at different levels of analysis and have complementary strengths. Dictionary learning (SAEs) operates at the feature level. It learns a decomposition of activation vectors into a large set of sparse, monosemantic feature directions. The strength of SAEs is scalability: once trained, a SAE can decode any activation vector into features. The weakness is that SAEs provide observational analysis, not causal analysis. Knowing which features are active does not tell you which features caused a given output. Circuit analysis operates at the connectivity level. It identifies which components (attention heads, MLP neurons) participate in a specific behavior and how they are connected. The strength of circuit analysis is causal rigor: activation patching establishes necessary and sufficient components. The weakness is scalability and behavioral specification — you need to know what behavior to investigate, and the investigation is expensive for large models. Representation geometry operates at the spatial level. It analyzes the geometric structure of activation space — how concepts cluster, what relationships are encoded as vector offsets, how concepts are linearly separable. The strength of geometry is its theoretical elegance and its connection to neuroscience. The weakness is that geometric relationships do not always correspond to causal roles — a concept can be geometrically distinct from another without being causally responsible for different outputs. Together, these three methods provide a more complete picture than any one alone: geometry tells you what representations exist, SAEs tell you which features activate for which inputs, and circuit analysis tells you which components causally implement which behaviors.
Q: What does the "compositionality gap" mean for the long-term prospects of mechanistic interpretability, and how might the field address it?
A: The compositionality gap is the observation that current interpretability tools explain individual features and circuits but struggle to explain how features compose into complex, multi-step reasoning. A SAE can identify that a model has separate features for "addition," "comparison," "negation," and "output formatting." But it cannot yet explain how these features interact during a multi-step arithmetic problem. The gap is fundamental because transformer computation is compositional — the power of transformers comes precisely from composing simple operations into complex behaviors. If interpretability can only explain the simple operations, it misses the most important part of how the model works. Addressing the compositionality gap likely requires new methods that track feature interactions across layers and time steps — not just which features are active but how their activation propagates through the computational graph. Some researchers are exploring dynamic circuit analysis that tracks feature causality through the forward pass. Others are developing higher-order SAEs that decompose not just activations but activation changes (gradients, Hessians). A complete theory of compositionality in transformers may require theoretical advances connecting the transformer's architectural inductive biases to the compositional structure of natural language. As of 2026, this remains one of the field's most important open problems.
Q: What is "scalable interpretability," why does it matter, and what are the most promising approaches to achieving it?
A: Scalable interpretability is the goal of providing interpretability that does not require compute proportional to the model being interpreted. Current methods fail this test: training a SAE on a GPT-4-class model costs hundreds of thousands of dollars. Running circuit analysis across all components of a 70B-parameter model is computationally prohibitive for any research group. Yet the models that most need interpretability — frontier models — are exactly the ones where interpretability is most expensive. Several promising approaches are being explored. First, transfer learning of interpretability: training SAEs on small models and applying the learned feature dictionary to large models, under the assumption that feature geometry is partially preserved across model sizes. Early results suggest some transfer is possible, though features in large models have richer structure that small-model SAEs cannot capture. Second, sparse coding theoretical bounds: theoretical work establishing that sparse feature representations can be learned with sublinear compute under certain sparsity and independence assumptions. Third, automated circuit discovery: using gradient-based methods to identify circuits automatically rather than through manual analysis. Fourth, probing with auxiliary models: training separate small models to predict features or circuit behaviors in larger models, effectively using the small model as a proxy interpreter. None of these approaches has yet demonstrated full scalable interpretability for frontier models, but each represents a credible research direction that the field is actively pursuing.
Q: How should the AI research community prioritize interpretability research relative to other AI safety approaches, and what would "success" look like in the next 3–5 years?
A: Interpretability research should be prioritized as a complementary tool within a broader safety portfolio — not as a standalone solution but as one of the highest-value additions to the toolkit. Its unique value is causal specificity: unlike behavioral testing, interpretability can identify which internal components drive which behaviors. This is irreplaceable for auditing models that may be deceptively competent in safe-looking contexts. Relative to other approaches (constitutional AI, RLHF, synthetic data generation), interpretability is less mature but more potentially transformative if it scales. Success in 3–5 years should be measured by concrete milestones: SAEs trained on frontier models at tractable cost, with publicly released feature dictionaries; automated circuit discovery that identifies known circuits (e.g., induction heads) in novel models without manual specification; demonstration that feature suppression can robustly shift model behavior across contexts in ways that generalize; and at least one documented case where interpretability analysis detected a safety-relevant property (deceptive reasoning, hidden goals, capability elevation) that behavioral testing missed. If the field can achieve these milestones by 2029, interpretability will be a routine part of the AI safety evaluation toolkit. If not, the field should honestly assess whether the current approach is on a productive trajectory or requires a fundamentally different paradigm.