Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligence

arXiv:2608.120362026-08-11

An AI-scientist system that autonomously investigates why AI models behave the way they do

Mechanist is an agentic system that generates hypotheses about how AI models work internally, runs experiments to test them, and iterates, largely replacing the manual work interpretability researchers currently do by hand. Built on a 13,000-paper interpretability knowledge graph, a 43-million-paper multidisciplinary database, and a library of 32 analysis methods, it outperformed Claude Code and existing AI-scientist systems on hypothesis quality and experimental reliability in benchmark tests. It went on to uncover a cross-modal safety risk, map out how models represent belief, use that mechanism to boost accuracy, and steer a DNA-generation model toward a target protein structure.

METAL MEDIA explanatory visual

Mechanist's four-stage autonomous research loop

Evidence statusMeasured results reported

Hypothesis generationDraws on a 13,000-paper interpretability knowledge graph and a 43-million-paper, 26-field database to propose hypotheses about model behavior
Experiment executionUses a library of 32 mechanistic analysis and intervention methods to design, implement, and run the experiment
VerificationChecks whether the experiment is valid and whether conclusions hold under different methods, datasets, or models
IterationRevises the hypothesis or experiment when verification fails and re-runs until results are reliable or the revision budget runs out

An explanatory diagram made by METAL MEDIA, not a figure supplied by the paper's authors.

What they did

The authors built Mechanist, an agentic framework that automates the mechanistic interpretability research cycle in four stages: hypothesis generation, experiment execution, verification, and iteration, instead of relying on manual human effort.
It grounds hypothesis generation in a purpose-built interpretability knowledge graph of about 13,000 papers plus a broader database of 43 million papers across 26 fields, and executes experiments using a curated library of 32 mechanistic-analysis methods.
In a benchmark that reproduced 16 existing papers, Mechanist scored higher than Claude Code and an existing AI-scientist system on experimental reliability (judged by both human experts and an LLM judge) and on the novelty, impact, and testability of generated hypotheses.
Mechanist discovered that unsafe traits can transfer to a student model across modalities even when the training data looks entirely safe, and it localized specific attention heads in Pythia models that separately govern world knowledge, personal belief, and attributed belief.
It turned this belief mechanism into a real-time intervention that amplifies the relevant heads during inference to improve accuracy, and separately steered an internal feature in the Evo2 DNA-generation model to increase the predicted alpha-helical content of generated protein-coding sequences.

Fig. 1: The comparison between our Mechanist and existing AI Scientists. a, the aims of existing AI Scientists. b, the aims of our Mechanist. c, the detailed differences between our Mechanist and existing AI Scientists.

Fig. 2: Overview and evaluation of Mechanist. a, The Mechanist framework consists of four stages: hypothesis generation, experiment execution, verification, and iteration. Specifically, hypothesis generation is inspired by interdisciplinary knowledge from the existing knowledge library, to which new discoveries validated by Mechanist are subsequently added. b, Benchmark design for comparing Claude Code, AI-Scientist and Mechanist. The benchmark evaluates the reliability of experimental execution by reproducing the paper and assesses generated hypotheses for novelty, impact, and testability. c, Reliability of experimental execution assessed through the reproduction of 16 existing papers. Experimental outcomes are independently evaluated by human experts and by an LLM judge, Claude Opus 5. d, Agreement between human and LLM evaluations of experimental reliability. Each point represents one reproduced experiment; the dashed line indicates perfect agreement. e, Overall quality of hypotheses generated for tasks in science, language, reasoning, and safety. f, Evaluation of hypotheses generated by Claude Code, AI-Scientist, and Mechanist across three dimensions: novelty, impact, and testability. g, Joint novelty and testability space of the hypotheses generated by CC.

Table 1: Dataset statistics for belief-state evaluation. The analysis dataset is used for behavioural evaluation, mechanism localization, causal validation, and router training, while the proposition-disjoint test dataset is reserved for intervention evaluation. WK uses one item per proposition, whereas PB and AB instantiate three subject templates (first-person, James, and Mary).
Dataset	Category	# propositions	WK	PB	AB
Analysis	colour	34	34	102	102
	taxonomy	95	95	285	285
	geography	36	36	108	108
	math	22	22	66	66
	world	40	40	120	120
Analysis total	–	227	227	681	681
Test	colour	10	30	90	90
	taxonomy	15	45	135	135
	geography	14	42	126	126
	math	10	30	90	90
	world	9	27	81	81
	chemistry	12	36	108	108
	biology	15	45	135	135
	astronomy	12	36	108	108
	units	12	36	108	108
	medicine	40	40	120	120
Test total	–	149	367	1 101	1 101

Fig. 3: Mechanist extends subliminal learning to the transfer of opposing preferences in the multimodal setting. a, Evolution of subliminal learning research. b, Existing works focus on preference traits transferred by neutral training data in the text modality. A GPT-4.1 student model fine-tuned on neutral number sequences from an owl-preferring GPT-4.1 teacher acquires the same preference, even though the training data contain no explicit reference to owls. c, Behaviors discovered by Mechanist. Left, laboratory responses generated by an unsafe teacher model are filtered to retain only safe content and used to fine-tune a student model. Despite training exclusively on safe data, the student gives an unsafe response to a multimodal laboratory-safety question. Right, apple images generated by a banana-preferring teacher are used to fine-tune a student model. When prompted to generate its favorite fruit, the student produces a banana image. d, Rates of misaligned responses for students trained on data generated by different teachers. Left, unsafe-response rates for the untuned Qwen3.5-9B baseline and student models (initialized from Qwen3.5-9B) trained on data from a regular or unsafe teacher model. Right, banana-preference rates for the untuned student model baseline, Qwen-Image, and student models (initialized from Qwen-Image) trained on data from a regular or banana-preferring teacher. Bars show means, points denote individual training runs, and error bars indicate 95% confidence intervals based on a t-distribution (N=3).

Fig. 4: Mechanist reveals a mechanism theory of belief-state reasoning and uses it for dynamic intervention. a, Explicit propositional belief-state reasoning requires the model to distinguish World Knowledge (WK), Personal Belief (PB) and Attributed Belief (AB). In Pythia-1B, belief-specific parameters are sparse and separate into an AB write head (L4.H1) and PB correction heads (L9.H1, L7.H5 and L12.H1). b, Formation of belief heads during Pythia-1B pretraining. AB emerges before PB. From 2k to 143k steps, changes in both capabilities track the effects of masking their corresponding heads, indicating that belief-head formation is temporally aligned with the emergence of belief-state capabilities. c, Dynamic intervention based on the discovered mechanism. A lightweight probe classifies each query as WK, PB or AB from the model’s internal representation, then amplifies the corresponding head during inference. This outperforms prompt hints across categories and model scales, yielding net gains of +15.3%, +8.8% and +3.5% for Pythia-410M, Pythia-1B and Pythia-2.8B, respectively.

Table 2: Prompt templates for WK, PB, and AB. Bracketed text denotes an item-specific field.
Frame	Template	Target
WK	[Factual cloze question] Answer:	yfact
PB	[Subject] believes [conflicting proposition]. In reality, [factual cloze question] Answer:	yfact
AB	[Subject] believes [conflicting proposition]. [Subject] thinks [belief cloze question] Answer:	ybelief

Fig. 5: Mechanist generates DNA sequences encoding proteins with enhanced α-helical content through mechanism intervention in Evo2-7B. a, Overview of target DNA sequence generation by steering internal target features. Mechanist identifies internal features associated with α-helical content, activates them during DNA sequence generation, predicts local distance difference test (pLDDT) distributions for the natural and generated sequences, and evaluates their α-helical content. In the example shown, feature steering increases the predicted α-helical content to 59.1% while maintaining structural quality with a pLDDT score of 0.79. b, Mean α-helical content across 900 generated sequences. Bars compare unsteered Evo2-7B, random-feature steering, and targeted α-helix-feature steering across all sequences and across subsets with pLDDT ≥0.4 or pLDDT ≥0.5. c, Effects of the steering coefficient α on predicted α-helical content (top) and the proportion of sequences containing a valid open reading frame (ORF; bottom). Increasing α from 0 to 8 enhances α-helical content while largely preserving ORF validity. Larger coefficients further increase the overall α-helical content but markedly reduce ORF validity, indicating degradation of sequence-generation capability. We therefore select α=8, which yields the highest α-helical content among valid sequences. d, Representative predicted structures generated from two input DNA sequences using steering coefficients of α=0, 2, 4 and 8. Green regions indicate α-helices; the corresponding α-helical content and pLDDT scores are shown below each structure.

Fig. 6: The overview of our interpretability database.

Table 3: Cross-model belief-state results. WK denotes world-knowledge recall, PB denotes factual judgement under a conflicting belief context, and AB denotes reporting the attributed belief.
Model	WK	PB	AB	Mechanistic result
Pythia-410M	0.881	0.852	0.461	Weak AB; no clean localization
Pythia-1B	0.925	0.786	0.833	AB L4.H1; PB {L9.H1, L7.H5, L12.H1}
Pythia-2.8B	0.960	0.994	0.794	AB L5.H22; PB top-25 heads
OLMo-1B	0.930	0.766	0.727	AB L2.H11; PB {L9.H9, L11.H9, L9.H0}
OLMo-7B	0.943	0.987	0.764	AB L2.H7; PB top-25 heads
GPT	1.000	1.000	0.896	Closed-source; behaviour only

Fig. 7: Overview of our interpretability database from the perspective of interpretable objects.

Fig. 8: Overview of our interpretability database from the perspective of application scenarios.

Findings

Across a benchmark reproducing 16 existing papers, both human experts and an LLM judge (Claude Opus 5) rated Mechanist's experimental execution as more reliable than Claude Code and an existing AI-scientist system.
A student model fine-tuned on training data filtered to remove all unsafe content still gave unsafe responses to multimodal lab-safety questions at a rate of 48.6%, compared with 20.3% for the untuned baseline and 18.3% for a student trained on a regular teacher's safe data.
A student model fine-tuned on images with all bananas filtered out still generated bananas 25.6% of the time when asked for its favorite fruit, compared with 2.5% for the untuned baseline and 2.1% for a regular-teacher control.
In Pythia-1B, ablating specific heads (L4.H1 for attributed belief; L9.H1, L7.H5, L12.H1 for personal belief) sharply reduced accuracy on their respective belief tasks while leaving other tasks and general language-model perplexity largely unaffected, confirming these heads' distinct causal roles.
Amplifying the relevant belief heads at inference time using a lightweight probe produced larger accuracy gains than prompt-hint baselines (+15.3%, +8.8%, +3.5% for Pythia-410M/1B/2.8B respectively), and steering a target feature in Evo2-7B raised mean predicted alpha-helical content from 43.8% to 56.6% across 900 generated sequences.

Fig. 9: Mechanism methods for large language models and multi-modal models.

Fig. 10: Reliability of reproduced claims across research areas. Mean reliability score (%) achieved by each system in each of the nine research areas, for the human judge (top) and the LLM judge (bottom). Bars show the mean and error bars denote 95% confidence intervals.

Where it can be used

Safety auditing before deployment, to check whether models trained on seemingly safe data can still inherit hidden unsafe tendencies across modalities.
Real-time inference-time correction for models that confuse their own world knowledge with beliefs attributed to other people or contexts.
Mechanism-guided design of scientific foundation models, such as steering internal features of DNA- or protein-generation models toward desired structural properties.
As a research assistant that autonomously proposes and tests hypotheses about newly observed AI behaviors or risks, generating experiments for human researchers to review.

Fig. 11: Reliability of reproductions along the four evaluation dimensions. Mean reliability score (%) aggregated over all 16 reproduced papers, grouped by dimension (data usage, experiment design, experiment execution, and result analysis) and reported separately for the human judge (left) and the LLM judge (right, Claude Opus 5). Bars show the mean and error bars denote 95% confidence intervals.

Fig. 12: Mechanist is the most reliable system under every judge, and the judges agree with one another. Three human experts, Claude Opus 5, and GPT-5.6-sol independently scored the same 48 system-paper units (16 papers × 3 systems) with an identical reliability rubric. a, Mean reliability score (%) of each system under each judge. Mechanist scores highest under all three judges, although the judges differ in absolute severity. Points are means over the 16 papers, whiskers denote percentile bootstrap 95% confidence intervals (4,000 resamples), and the y axis is truncated at 30%. b, Paired scores for the three judge pairs (left, Claude Opus 5 versus GPT-5.6-sol; middle, human versus Claude Opus 5; right, human versus GPT-5.6-sol). Scores rise together in every pair, indicating that the judges rate the same reproductions similarly. Each point is one system-paper unit coloured by system, the dashed line is a through-origin least-squares fit, and Spearman ρ is the rank correlation.

Limits and open work

The belief-mechanism experiments relied mainly on Pythia and OLMo, whose intermediate pretraining checkpoints are public; closed models like GPT could only be tested behaviorally, without internal analysis.
The cross-modal risk-transfer experiments were limited to specific model pairs (Qwen3.5-9B, Qwen-Image) and specific domains (lab safety, fruit preference), so generalization to other models or domains is untested.
The DNA-sequence intervention was demonstrated only for alpha-helical content in Evo2-7B; extension to other biological properties or other foundation models is not yet reported.
Increasing the steering coefficient too far degrades the model's ability to generate valid sequences (open reading frames), showing a trade-off that requires careful coefficient selection.
The belief intervention depends on a lightweight probe correctly classifying each query's frame (world knowledge, personal belief, or attributed belief); the impact of probe misclassification is not separately analyzed.

Why it matters

As AI systems advance faster than our ability to understand them, this kind of system offers a way to automate the search for hidden mechanisms and risks rather than relying solely on manual expert analysis. The demonstrated case studies show concrete, if narrow, paths from mechanistic discovery to safety auditing, model improvement, and scientific design.

Terms in this paper

interpretability · research aimed at understanding why an AI model produces the outputs it does, by examining its internal workings
subliminal learning · a phenomenon where a trait transfers from a teacher model to a student model through training data that appears unrelated to that trait
attention head · a small computational unit inside a transformer model that selectively processes certain information
sparse autoencoder (SAE) · a tool that decomposes a model's complex internal representations into individually interpretable features
pLDDT · a confidence score for predicted protein structure accuracy

Original abstract (English)

AI models have achieved remarkable success across diverse domains, yet the mechanisms underlying their capabilities and the risks they may pose remain poorly understood. As AI development becomes faster and increasingly automated, mechanistic exploration remains largely manual, widening the gap between what models can do and our ability to understand and control them. To bridge this gap, we introduce Mechanist, an agentic system that uses AI as a scientific instrument for the autonomous discovery of mechanisms underlying AI intelligence. To support autonomous mechanistic discovery, we construct an interpretability-focused knowledge graph of approximately 13,000 papers and integrate it with a multidisciplinary database of 43 million papers spanning 26 fields. We further curate a library of 32 foundational methods for mechanism analysis, causal intervention, and validation. Compared with Claude Code and existing AI-scientist systems, Mechanist generates more valuable mechanism hypotheses and executes experiments more reliably. Mechanist also demonstrates a progression from discovering model behaviors to explaining and controlling AI models. Specifically, Mechanist first uncovers a counterintuitive safety risk in scientific laboratories, showing that unsafe traits can transfer across modalities through apparently safe training data. Mechanist then develops a mechanism theory of belief, revealing how models represent world knowledge, form beliefs, infer the beliefs of others, and how these mechanisms emerge during pretraining. Finally, Mechanist translates these mechanistic insights into practical interventions that improve model performance across diverse scenarios and steer scientific foundation models toward generating DNA sequences with specified properties.

Authors · Mengru Wang

Read on arXiv

Latest papers

All papers →

Latest from METAL MEDIA

Figures: Mengru Wang et al., arXiv:2608.12036, arxiv-nonexclusive