Everything K-culture — comebacks to K-beauty, straight to your inboxGet it in your inbox›
Scaling Inherently Interpretable Language Models
arXiv:2608.075942026-08-05
A study showing that building interpretability into training itself, instead of explaining models after the fact, doesn't hurt performance and actually gets easier as models scale up
Instead of training a language model and reverse-engineering it afterward, this work bakes an 'explainability' requirement directly into the training objective from the start. The team built a 33,000-plus concept library called Atlas and a diffusion language model, Steerling-8B, that uses it, testing across three orders of magnitude of compute. Interpretability metrics actually improved with scale, and Steerling-8B came within roughly 10% of the average benchmark performance of open peer models trained on 2-16x more compute.
METAL MEDIA explanatory visual
Building interpretability into the training pipeline
Evidence statusMeasured results reported
Atlas pipelineDocuments are chunked and tagged (Stage 1), tags are clustered into 33,732 canonical concepts (Stage 2), and an annotator model is trained to label text at scale (Stage 3).
Concept bottleneckAn additive concept module sits between Steerling-8B's transformer backbone and its output head, exactly decomposing output logits into known-concept, unknown-concept, and residual contributions.
Three attributionsFor any output, the model computes input token attribution, concept attribution, and training data (similarity) attribution.
Scaling sweepsIsoFLOP experiments across three orders of magnitude of compute compare autoregressive and diffusion models with and without the concept module, showing a fixed small capability cost and improving interpretability metrics with scale.
Concept steeringAdding or subtracting a learned concept embedding direction from hidden states amplifies or suppresses that concept in generated output, without retraining.
An explanatory diagram made by METAL MEDIA, not a figure supplied by the paper's authors.
What they did
Prior interpretability work analyzes a model only after it's fully trained, using tools like probes, feature attribution, or sparse autoencoders, but the paper argues these can't guarantee the model actually uses the information they find.
Instead, the authors build interpretability conditions into data curation, architecture, and loss design from the start, defining three attribution axes: input tokens, human-understandable concepts, and training data.
They built Atlas, a pipeline that tags millions of documents, clusters the tags into 33,732 canonical concepts, and trains an annotator model that has labeled over 1 trillion tokens.
Steerling-8B, a diffusion language model with block-causal attention, has an additive concept bottleneck inserted between its transformer backbone and output head so that output logits decompose exactly into concept contributions.
In IsoFLOP scaling sweeps across three orders of magnitude of compute, adding the concept module caused only a small fixed performance offset, while interpretability metrics improved as models scaled up.
Table 1: Transformer and language-model notation.
\topruleSymbol
Meaning
\midrule𝐱=(x1,…,xN)
Input token sequence of length N
xi
Token at position i
V
Vocabulary
Txi∈ℝd
Learned embedding of token xi
d
Hidden dimension
L
Number of transformer layers
h∈ℝd
Transformer hidden state at a given position
W∈ℝ|V|×d
Language modeling head
Wy
Row of W corresponding to token y
ℓy=h⊤Wy
Logit for output token y
pθ
Model with parameters θ
𝐱<i
Sub-sequence of tokens before position i
ℒ\textAR
Autoregressive training loss
\midrule𝐱t
Corrupted sequence at noise level t
t∈[0,1]
Noise level
M(𝐱t)
Set of masked positions in 𝐱t
ℒ\textMDM
Masked diffusion training loss
\bottomrule
Figure 1: The recipe for training an interpretable model.
Table 2: Removing any single recipe modification breaks a specific condition of Definition 3.3.
\topruleRemove…
Condition broken
Consequence
\midruleConcept annotations
Interpretation (4)
No semantic targets
Bottleneck architecture
Nativeness (1), Agreement (2)
ϕc(y) not computable
Trained absence baseline
Validity (3)
Input-attribution baselines OOD
Interpretability losses
Coverage (5), Interpretation (4)
Residual absorbs capacity
\bottomrule
Figure 3: Overview of the Atlas three-stage annotation pipeline: documents are chunked and tagged (Stage 1), tags are clustered and canonicalized into a concept library (Stage 2), and a concept annotator model is trained for scalable text annotation (Stage 3).
Table 4: Phase 2 blind fit scores by candidate type. Human annotators rated how well each candidate name fit the same lifted-word evidence. Counts sum to 1,025 individual ratings.
\toprulecandidate type
mean fit (1–5)
sd
n
\midruleLLM (Atlas) label
3.98
1.04
205
human label
3.50
1.17
402
embedding distractor
2.72
1.30
205
taxonomy distractor
2.08
1.11
205
filler (floor)
1.62
1.06
008
\bottomrule
Figure 5: Stage 1 tag validation scores. Distributions of average tag-relevance ratings on a 1–5 scale: per-chunk averages (left column; each chunk’s tag ratings averaged) and per-tag averages (right column; each unique tag’s ratings averaged over the chunks it appears in), as rated by the LLM judge over 2.93M sampled chunks (top row) and by three human annotators on a 50-chunk sample (bottom row).
Table 5: Interpretability metrics. ↑ or ↓ indicates the direction of better performance.
\topruleMetric
Direction
What it measures
\midruleConcept Loss
↓
Does the concept module detect the right concepts?
Concept Independence Loss
↓
Are known and unknown heads disentangled?
Concept Contribution
↑
Do predictions route through the concept module?
Known Concept Alignment
↑
Do concept embeddings point at related tokens?
\bottomrule
Figure 6: Comparison of all-mpnet-base-v2 and Qwen3-Embedding-0.6B embeddings for clustering ∼14M LLM-generated tag annotations via the k-means clustering algorithm. We evaluate Silhouette score (left plot), Davies–Bouldin index (middle plot), and coherence score (right plot) on randomly sampled cluster members. As k increases, all-mpnet-base-v2 consistently yields more coherent clusters than Qwen3-Embedding-0.6B.
Table 6: Compute-optimal scaling exponents and irreducible-loss asymptotes. Subscripts are 90% bootstrap confidence intervals.
\topruleModel
αP
αD
αL
ℒ∞
\midrule Ours (Autoregressive)
AR
0.528[−0.025,+0.023]
0.445[−0.036,+0.114]
−0.060[−0.002,+0.002]
1.857[−0.335,+0.071]
AR+Concept
0.621[−0.041,+0.077]
0.524[−0.112,+0.108]
−0.070[−0.002,+0.003]
1.193[−0.703,+0.394]
\midrule Ours (Causal Diffusion)
CDLM
0.632[−0.091,+0.075]
0.481[−0.111,+0.162]
−0.053[−0.004,+0.004]
2.658[−0.708,+0.181]
CDLM+Concept
0.503[−0.046,+0.051]
0.374[−0.170,+0.174]
−0.072[−0.003,+0.003]
1.942[−1.805,+0.529]
\midrule Masked diffusion (prior)
96
0.514
0.486
—
2.41
146
0.566[−0.022,+0.019]
0.434[−0.019,+0.020]
−0.0496[−0.0004,+0.0003]
—
\midrule Autoregressive (prior)
54
0.490
0.510
—
1.69
124
0.464
0.536
—
—
18
0.524
0.476
—
—
\bottomrule
Figure 7: Stage-2 transformation of noisy LLM tags into a canonical concept: raw tags are embedded and clustered, incoherent clusters are filtered out, each surviving cluster is labeled into a single concept (name with italic description), and semantically adjacent concepts are merged by cosine similarity into one canonical entry.
Table 7: Interpretability scaling fits. Subscripts on e are 90% bootstrap confidence intervals. The 8B columns compare the small-scale extrapolation against the actual Steerling values.
\topruleFamily
β
e [90% CI]
R2
8B pred
8B actual
Δ
\midrule Concept Loss
AR+Concept
0.385
0.002[−0.002,+0.000]
0.704
0.002
—
—
CDLM+Concept
0.054
0.000[−0.000,+0.002]
0.648
0.002
0.002
+0.000
\midrule Concept Independence Loss
AR+Concept
0.279
1.310[−1.310,+0.000]
0.505
1.550
—
—
CDLM+Concept
0.217
0.000[−0.000,+3.687]
0.749
2.157
1.550
−0.611
\midrule Concept Contribution
AR+Concept
0.212
0.937[−0.047,+0.063]
0.630
0.915
—
—
CDLM+Concept
0.134
1.000[−0.149,+0.000]
0.496
0.915
0.876
−0.039
\midrule Known Concept Alignment
AR+Concept
0.099
5.000[−1.190,+0.000]
0.557
4.200
—
—
CDLM+Concept
0.349
3.920[−0.135,+1.080]
0.637
3.870
3.770
−0.100
\bottomrule
Figure 8: UMAP projection of a subsample of concept embeddings across five taxonomy groups. The cards showcase representative concepts from distinct regions of the embedding space to illustrate local semantic clusters.
Table 8: Joint interpretability scaling fits, m(P,D)=e±(APP−α+ADD−β), with α the parameter exponent and β the token exponent. Subscripts on e are 90% bootstrap confidence intervals. The 8B columns compare the small-scale extrapolation against the actual Steerling values.
\topruleFamily
α
β
e [90% CI]
R2
8B pred
8B actual
Δ
\midrule Concept Loss
AR+Concept
0.093
0.767
0.000[−0.000,+0.001]
0.945
—
—
—
CDLM+Concept
0.427
1.255
0.002[−0.000,+0.000]
0.893
0.002
0.002
+0.000
\midrule Concept Independence Loss
AR+Concept
1.085
0.529
1.347[−1.347,+0.745]
0.624
—
—
—
CDLM+Concept
0.904
0.659
0.000[−0.000,+1.908]
0.937
0.624
1.550
+0.921
\midrule Concept Contribution
AR+Concept
0.541
0.739
0.903[−0.028,+0.097]
0.784
—
—
—
CDLM+Concept
0.441
2.077
0.864[−0.012,+0.018]
0.883
0.844
0.876
+0.032
\midrule Known Concept Alignment
AR+Concept
2.660
0.154
4.640[−0.363,+0.360]
0.926
—
—
—
CDLM+Concept
0.022
0.346
5.000[−1.125,+0.000]
0.933
3.967
3.770
−0.194
\bottomrule
Figure 9: Stage 2 concept validation scores. Distributions of average concept-relevance ratings on a 1–5 scale: per-chunk averages (left column) and per-concept averages (right column; each concept’s ratings averaged over the chunks it is assigned to), as rated by the LLM judge over 2.93M chunks (top row) and by human annotators on a 50-chunk sample (bottom row).
Table 9: Steerling-8B pretraining corpus. Token counts are post-tokenization.
\topruleSource
Documents
Chunks
Tokens
\midruleNemotron-CC-HQ (real)
740M
5.1B
547B
Nemotron-CC-HQ (synthetic)
971M
4.8B
498B
peS2o
38.8M
565.4M
59B
arXiv
3.9M
142.2M
20.4B
Wikipedia & Wikibooks
6.1M
36.8M
3.8B
OpenWebMath
2.9M
76.8M
12.1B
Algebraic Stack
2.8M
65.6M
12.1B
StarCoder
78.6M
317M
91.4B
\midruleTotal
1.84B
11.1B
1.24T
\bottomrule
Figure 10: Stage 3 annotator validation on held-out chunks. Distributions of average predicted-concept relevance ratings from the LLM judge: per-chunk averages (left) and per-concept averages (right).
Table 10: Steering results on random concepts of the pretrained Steerling-8B. Steered denotes layer injection.
\topruleMethod
Concept ↑
Quality ↑
Harmonic ↑
\midruleUnsteered
0.033
1.108
0.065
Prompting
0.908
1.588
1.156
Steered
1.072
0.972
1.020
\bottomrule
Figure 11: Schematic of the LCC-derived taxonomy used to organize the Atlas concept library. The 33,732 canonical concepts are mapped onto approximately 2,600 occupied taxonomy nodes. The figure shows the root structure and expands the Science (Q) branch to illustrate how top-level classes decompose into more specific areas such as Mathematics (QA), Physics (QC), and their subdivisions; other branches are collapsed for readability.
Table 11: Data compositions compared in the mid-training ablation. Each arm is a 10B-token run from the final pretraining checkpoint of 1.2T tokens; entries are token counts in billions.
\topruleSource
Math-heavy
Balanced
Code-augmented
Code-only
\midruleNemotron (real)
5.0B
4.7B
5.0B
–
Dolmino Math
5.0B
2.1B
2.0B
–
StarCoder
–
–
1.0B
10.0B
FLAN
–
1.7B
1.0B
–
peS2o
–
0.6B
0.4B
–
Wikipedia & Wikibooks
–
0.7B
0.5B
–
Stack Exchange
–
0.2B
0.2B
–
\midruleTotal
10.0B
10.0B
10.0B
10.0B
\bottomrule
Figure 12: Distribution of over 33,000 concepts across the top-level LCC classes and notable subclasses. Science (Q) accounts for the largest share, followed by Technology (T), Social Sciences (H), and Medicine (R); all root classes are represented to varying degrees. The distribution mirrors the composition of the pretraining corpus rather than a curatorial choice.
Table 12: Downstream performance of the four mid-training compositions, each a 10B-token run from the final pretraining checkpoint, against the base model. Best in each column in bold. HSwag: HellaSwag; HEval: HumanEval; WinoG: WinoGrande.
\topruleComposition
MMLU
GSM8K
ARC-C
HSwag
HEval
MBPP
WinoG
Avg.
\midrulePretrained model
0.298
0.140
0.484
0.673
0.049
0.004
0.596
0.321
\midruleMath-heavy
0.376
0.441
0.492
0.681
0.037
0.012
0.616
0.379
Balanced
0.416
0.328
0.497
0.693
0.037
0.006
0.616
0.371
Code-augmented
0.416
0.328
0.498
0.693
0.055
0.012
0.628
0.376
Code-only
0.303
0.086
0.434
0.630
0.061
0.012
0.583
0.301
\bottomrule
Figure 13: Phase 2 human-evaluation fit scores by candidate type. Human annotators rated, on a 1–5 scale, how well each candidate name fit the same lifted-word evidence for a concept. Points show mean fit scores and horizontal bars show 95% confidence intervals; the dotted vertical line marks the neutral midpoint of the scale. Atlas labels score comparably to or above independently generated human labels, and both are clearly separated from embedding, taxonomy, and filler distractors, indicating that raters were not merely assigning high scores to any plausible label.
Table 13: The final Steerling-8B midtraining mixture of 150B tokens. Dolmino Math is upsampled roughly twofold. Proportions follow the code-augmented composition of \creftab:midtraining-composition.
\topruleSource
Number of Tokens
Ratio (%)
\midruleNemotron (real)
72.79B
48.5
StarCoder
30.75B
20.5
peS2o
21.98B
14.7
Dolmino Math (∼2×)
16.05B
10.7
FLAN
6.38B
4.3
Wikipedia & Wikibooks
1.50B
1.0
Stack Exchange
0.56B
0.4
\midruleTotal
150B
100.0
\bottomrule
Table 14: Masking schedule ablation, each a 10B-token run from the final pretraining checkpoint. Best in each column in bold. HSwag: HellaSwag; HEval: HumanEval; WinoG: WinoGrande.
\topruleMasking
MMLU
GSM8K
ARC-C
HSwag
HEval
MBPP
WinoG
Avg.
\midrulePretrained model
0.298
0.140
0.484
0.673
0.049
0.004
0.596
0.321
50% uniform
0.416
0.328
0.498
0.693
0.055
0.012
0.628
0.376
80% Gaussian
0.280
0.355
0.500
0.686
0.055
0.008
0.628
0.359
\bottomrule
Table 15: Steering ablation: Adding the steering phases improves every metric on the steering benchmark (150), while LM Harness performance stays mostly unchanged.
\toprule
LM Harness ↑
Steering benchmark ↑
\cmidrule(lr)2-6 \cmidrule(lr)7-9
MMLU
GSM8K
ARC-C
HSwag
WinoG
Concept
Quality
Harmonic
\midruleMath-heavy
0.370
0.431
0.499
0.682
0.618
1.208
0.989
1.088
+ steering
0.384
0.415
0.505
0.681
0.611
1.244
1.139
1.189
\bottomrule
Table 16: Steerling-8B before and after mid-training across the LM Harness suite (accuracy, %). HSwag: HellaSwag; HEval: HumanEval; WinoG: WinoGrande.
\toprule
MMLU
GSM8K
ARC-C
HSwag
HEval
MBPP
WinoG
Avg.
\midrulePretrained
29.8
14.0
48.4
67.3
4.9
0.4
59.6
32.1
Mid-trained
46.4
44.4
52.3
70.3
8.5
11.0
64.2
42.4
\bottomrule
Table 17: Interpretability metrics for Steerling-8B before and after mid-training.
\toprule
Pretrained
Mid-trained
\midrule Concept Loss ↓
0.002
0.002
Concept Independence Loss ↓
1.907
1.546
Concept Contribution ↑
0.851
0.876
Known Concept Alignment ↑
3.730
3.770
\bottomrule
Table 18: Steering benchmark scores for the pretrained and mid-trained Steerling-8B checkpoints. Mid-training improves every steering metric.
\topruleCheckpoint
Concept ↑
Quality ↑
Harmonic ↑
Sample harmonic ↑
\midrulePretrained (1.2T)
1.072
0.972
1.020
0.843
Mid-trained (1.35T)
1.247
1.064
1.148
0.963
\bottomrule
Table 19: Steerling-8B base model against open base models of comparable size. HSwag: HellaSwag; WinoG: WinoGrande. Values marked ∗ are taken from the OLMo 3 report (100); all other peer numbers are from the respective model reports.
\topruleModel
HSwag
WinoG
PIQA
MMLU
ARC-C
GSM8K
Math
Avg.
\midruleSteerling-8B
70.3
64.2
75.9
46.4
52.3
44.4
8.0
51.6
\midruleLLaMA2 7B
76.0
72.5
79.1
45.9
46.3
13.1
4.3
48.2
DeepSeek 7B
75.4
70.5
79.2
48.2
48.1
17.4
6.0
49.3
Gemma 1 7B
81.2
72.3
81.2
64.3
53.2
46.4
24.3
60.4
LLaDA 8B
70.5
74.8
73.6
65.9
45.9
70.3
31.4
61.8
LLaMA3 8B
79.1
77.3
80.6
65.4
53.1
48.7
16.0
60.0
OLMo 2 7B
83.8
77.2
80.1∗
63.7
79.8
67.5
19.1∗
67.3
\bottomrule
Table 20: Concept module notation, grouped by role.
\topruleSymbol
Type
Meaning
\midrule Hidden states
h
vector
Transformer hidden state
h¯
vector
Bottlenecked state passed to the LM head
h¯=k^+u^+ε
equation
Concept module decomposition
\midrule Concept module heads
f
function
Known head
g
function
Unknown head
k=σ(f(h))
vector
Per-concept activation probabilities, known
u=σ(g(h))
vector
Per-concept activation probabilities, unknown
k\textknown
scalar
Top-k count for the known head
k\textunknown
scalar
Top-k count for the unknown head
\midrule Concept embeddings
K
matrix
Known concept embedding matrix
U
matrix
Unknown concept embedding matrix
Ki
vector
Embedding of known concept i
Uj
vector
Embedding of unknown concept j
n
scalar
Number of known concepts
m
scalar
Number of unknown concepts (m≫n)
R
scalar
Factorization rank of unknown embedding matrix
\midrule Concept contributions
k^=∑ikiKi
vector
Known concept contribution
u^=∑jujUj
vector
Unknown concept contribution
ε=h−k^−u^
vector
Residual term
\midrule Logit decomposition
Wy
vector
Row of the LM head for output token y
ℓy
scalar
Output logit for token y
\bottomrule
Table 21: Concept module notation, grouped by role.
Table 22: Robustness checks for the Phase 2 comparison between pipeline labels and independently generated human labels.
\topruleAnalysis
Result
\midruleMean Phase 2 fit score
LLM 3.98 vs. human 3.50
Top-two rating rate (≥4)
LLM 79% vs. human 63%
Bayesian cumulative-link model
OR 2.38, 95% CrI [1.23,4.01]
Posterior probability of LLM advantage
0.99
Paired comparison probability
0.62, bootstrap 95% CI [0.58,0.66]
Gaussian mixed model
+0.47 points, 95% CI [0.28,0.65]
\bottomrule
Table 23: Notation introduced in the scaling-law analysis, grouped by role: compute and resources, validation losses, power-law parameters, and interpretability scaling.
\topruleSymbol
Type
Meaning
\midrule Compute and resources
C
scalar
Total training FLOPs
M
scalar
Per-token FLOPs (forward + backward)
P
scalar
Non-embedding parameter count
D
scalar
Number of training tokens (total)
Di
scalar
Training tokens for the i-th checkpoint
\midrule Validation losses
ℒi
scalar
Measured validation loss for checkpoint i
ℒ∗(C)
function
Compute-optimal validation loss at budget C
ℒ∞
scalar
Irreducible validation loss
ℒ(P,D)
function
Joint Chinchilla loss surface
ℒ\textfit
function
Huber loss minimized in the joint fit
\midrule Power-law parameters
P∗(C)
function
Compute-optimal parameter count at budget C
aP,aL
scalars
Power-law coefficients (parameter and loss)
αP
scalar
Exponent on compute for parameter scaling
αL
scalar
Exponent on compute for loss scaling
αD
scalar
Exponent on compute for training-token scaling
AP,AD
scalars
Chinchilla coefficients (parameter and data sides)
α,β
scalars
Chinchilla exponents (parameter and data sides)
\midrule Interpretability scaling
m(C)
function
A metric as a function of compute
e
scalar
Irreducible value of a metric
A,β
scalars
Coefficient and exponent for metric scaling
\bottomrule
Table 24: Backbone architectures used across all four families. Backbone parameter counts exclude embeddings; +Concept totals include the concept module heads (concept classifier, and factorized unknown head concept embeddings are excluded). Sequence length is 4096 throughout.
\topruleSize
Layers L
Hidden d
Backbone params
+Concept total params
\midrule10M
6
320
9.2M
82.8M
25M
6
512
23.6M
110.2M
85M
10
768
86.5M
190.5M
200M
13
1024
197.7M
319.1M
400M
17
1280
401.1M
540.0M
800M
17
1792
779.9M
953.6M
1.5B
20
2304
1,510M
1,718M
3B
24
3072
3,228M
3,489M
5B
24
3840
5,694M
6,007M
\bottomrule
Table 25: IsoFLOP target compute budgets per family.
\topruleFamily
Slice 1
Slice 2
Slice 3
Slice 4
\midruleAR
6×1018
1019
3×1019
1020
CDLM
6×1018
1019
3×1019
1020
AR+Concept
1019
3×1019
1.1×1020
3.09×1020
CDLM+Concept
1019
3×1019
1.1×1020
3.09×1020
\bottomrule
Findings
Across IsoFLOP scaling sweeps spanning three orders of magnitude of compute, adding the concept module shifted compute-optimal scaling exponents by only a small, fixed offset that did not grow with scale.
In the same experiments, interpretability metrics such as concept prediction accuracy, separation of known versus unknown representations, the fraction of prediction routed through concepts, and alignment between concept embeddings and human labels all improved with increasing compute.
Extrapolating from small-scale fits using a joint Chinchilla-style scaling law predicted Steerling-8B's actual validation loss within 0.11 nats, and three of four interpretability metrics were predicted within tight bounds.
Steerling-8B, trained on 1.2 trillion pretraining tokens plus 150 billion midtraining tokens, landed within approximately 10% of the average benchmark performance of open peer models at similar parameter scale trained on roughly 2-16x more compute.
In Atlas pipeline validation, both an LLM judge and human annotators rated tag and concept relevance highly on average, and Atlas-generated concept labels scored comparably to or above independently written human labels while being clearly separated from embedding, taxonomy, and filler distractor labels.
Where it can be used
Tracing which input tokens, concepts, and training examples drove a given output could be used as an auditing tool for diagnosing model behavior.
Concept steering without retraining could be applied to adjust generation toward or away from a target concept at inference time.
Training data attribution could be tested for alignment fine-tuning, factual provenance tracing, or checking whether a model relies on proprietary fine-tuning data versus pretraining knowledge.
The three-stage Atlas pipeline (tagging, clustering, annotator training) could serve as a template for other groups needing concept annotation at web-scale corpora.
Limits and open work
Training data attribution is an approximate similarity retrieval in representation space, not a causal influence estimate, so it does not prove a training example actually caused an output.
The 33,732-concept library reflects the composition of this specific pretraining corpus (science, technology, social sciences, medicine, etc.), so it's unverified whether it generalizes to corpora with a very different domain or language mix.
Steerling-8B came within roughly 10% of peer average performance despite using 2-16x less compute, but gaps on individual benchmarks may still exist.
The concept module's parameter overhead (4% at 8B, projected to fall below 1% at frontier scale under the same parameterization) is an extrapolated estimate, not something directly measured on a frontier-scale model.
Human evaluation was based on a limited sample (a 50-chunk sample and 1,025 individual ratings), so broader human validation would still be needed.
Why it matters
If the claim holds, it challenges the long-standing assumption that interpretability and capability trade off against each other, opening a path to build large models that are auditable, correctable, and controllable by design rather than reverse-engineered after training. This could translate into practical tools for diagnosing model behavior, tracing it to training data, and correcting it via concept steering without retraining.
Terms in this paper
concept bottleneck · A layer inserted between a model's internal representations and its output that forces the output to decompose into human-interpretable 'concept' contributions
diffusion language model · A language model trained by masking parts of text and learning to reconstruct them, as opposed to predicting tokens one at a time left-to-right
post-hoc interpretability · Analyzing an already-trained model afterward with external tools to try to explain its behavior
IsoFLOP scaling · An experimental method that varies model size and data amount while holding total compute budget fixed, to study performance trends
Integrated Gradients · A method that computes token importance by integrating the gradient of the output along a path from a baseline input (here, the [MASK] token) to the actual input
Original abstract (English)
Interpretability is often treated as a tax on capability: language models are trained as opaque systems, then explained after the fact, with methods whose reliability is difficult to establish. In this work, we challenge this premise. Rather than reverse-engineering a model, we make interpretability a constraint of the training pipeline, optimized alongside the language modeling objective. Across three orders of magnitude of compute, on both autoregressive and diffusion language models, interpretability scales with capability rather than against it. Surprisingly, model representations become more disentangled and aligned with human-understandable concepts with scale.
We instantiate the training-time recipe with Steerling-8B, a diffusion language model with a causal attention mask. For any group of generated tokens, Steerling-8B attributes the output to relevant input tokens, human-understandable concepts, and training data. This enables closed-loop intervention: diagnose an output through its concept or feature attribution, retrieve similar training data, and correct the behavior through concept steering without retraining. Steerling-8B remains competitive with open peer models trained on substantially 2-16x more compute, suggesting a different scaling paradigm: interpretability can be designed into training, and it improves with scale.
