Scaling Inherently Interpretable Language Models

arXiv:2608.075942026-08-05

一项研究发现,把可解释性直接设计进训练过程,而不是训练完之后再事后分析,不仅不会拖累模型性能,规模越大反而越容易理解

这项工作没有采用先训练模型、再事后用探针等工具去反推解释的常见做法,而是把可解释性作为训练目标的一部分,从一开始就和语言建模目标一起优化。团队构建了包含三万三千多个概念的Atlas概念库,并据此训练出内置概念结构的扩散语言模型Steerling-8B,在跨越三个数量级的计算规模上做了实验。结果显示可解释性相关指标随规模增大反而变好,Steerling-8B的平均基准表现与用2到16倍计算量训练的同级公开模型相比,差距在约10%以内。

METAL MEDIA 解读图

把可解释性内建进训练流程的结构

证据状态已报告实测结果

Atlas流水线文档先被切块打标签(第一阶段),标签聚类归纳成三万三千七百三十二个规范概念(第二阶段),再训练出可大规模标注文本的标注模型(第三阶段)。
概念瓶颈结构在Steerling-8B的Transformer主干和输出层之间插入一个加法式概念模块,把输出logit精确分解为已知概念、未知概念和残差三部分的贡献。
三种归因针对任意输出,模型分别计算输入token归因、概念归因和训练数据(相似度)归因。
规模扫描实验跨越三个数量级计算量的IsoFLOP实验比较有无概念模块的自回归模型和扩散模型,显示性能代价保持较小且固定,而可解释性指标随规模增大持续改善。
概念操控(steering)将已学到的概念嵌入方向加到或从隐藏状态中减去,即可在不重新训练的情况下增强或抑制生成内容中的某个概念。

这是 METAL MEDIA 制作的解读图，并非论文作者提供的原图。

他们做了什么

以往的可解释性研究通常是模型训练完成后,用探针、特征归因、稀疏自编码器等工具事后分析,但论文指出这类方法无法保证模型真的在预测时使用了它们发现的信息。
作者转而把可解释性条件嵌入数据整理、架构设计和损失函数设计的每个环节,定义了输入归因、概念归因、训练数据归因三条解释路径。
团队构建了Atlas标注流水线:对上百万份文档打标签、聚类归纳成三万三千七百三十二个规范概念,并训练出一个标注模型,已为超过一万亿个token完成了概念标注。
扩散语言模型Steerling-8B采用分块因果注意力,并在Transformer主干和输出层之间插入一个加法式概念瓶颈层,使输出的logit可以被精确分解为各概念的贡献。
在跨越三个数量级计算量的IsoFLOP规模扫描实验中,加入概念模块只带来一个很小且固定的性能偏移,而可解释性指标反而随着模型规模增大而持续改善。

Scaling Inherently Interpretable Language Models figure 0

Table 1: Transformer and language-model notation.
\topruleSymbol	Meaning
\midrule𝐱=(x1,…,xN)	Input token sequence of length N
xi	Token at position i
V	Vocabulary
Txi∈ℝd	Learned embedding of token xi
d	Hidden dimension
L	Number of transformer layers
h∈ℝd	Transformer hidden state at a given position
W∈ℝ\|V\|×d	Language modeling head
Wy	Row of W corresponding to token y
ℓy=h⊤Wy	Logit for output token y
pθ	Model with parameters θ
𝐱<i	Sub-sequence of tokens before position i
ℒ\textAR	Autoregressive training loss
\midrule𝐱t	Corrupted sequence at noise level t
t∈[0,1]	Noise level
M⁡(𝐱t)	Set of masked positions in 𝐱t
ℒ\textMDM	Masked diffusion training loss
\bottomrule

Figure 1: The recipe for training an interpretable model.

Table 2: Removing any single recipe modification breaks a specific condition of Definition 3.3.
\topruleRemove…	Condition broken	Consequence
\midruleConcept annotations	Interpretation (4)	No semantic targets
Bottleneck architecture	Nativeness (1), Agreement (2)	ϕc(y) not computable
Trained absence baseline	Validity (3)	Input-attribution baselines OOD
Interpretability losses	Coverage (5), Interpretation (4)	Residual absorbs capacity
\bottomrule

Figure 3: Overview of the Atlas three-stage annotation pipeline: documents are chunked and tagged (Stage 1), tags are clustered and canonicalized into a concept library (Stage 2), and a concept annotator model is trained for scalable text annotation (Stage 3).

Table 4: Phase 2 blind fit scores by candidate type. Human annotators rated how well each candidate name fit the same lifted-word evidence. Counts sum to 1,025 individual ratings.
\toprulecandidate type	mean fit (1–5)	sd	n
\midruleLLM (Atlas) label	3.98	1.04	205
human label	3.50	1.17	402
embedding distractor	2.72	1.30	205
taxonomy distractor	2.08	1.11	205
filler (floor)	1.62	1.06	008
\bottomrule

Figure 5: Stage 1 tag validation scores. Distributions of average tag-relevance ratings on a 1–5 scale: per-chunk averages (left column; each chunk’s tag ratings averaged) and per-tag averages (right column; each unique tag’s ratings averaged over the chunks it appears in), as rated by the LLM judge over 2.93M sampled chunks (top row) and by three human annotators on a 50-chunk sample (bottom row).

Table 5: Interpretability metrics. ↑ or ↓ indicates the direction of better performance.
\topruleMetric	Direction	What it measures
\midruleConcept Loss	↓	Does the concept module detect the right concepts?
Concept Independence Loss	↓	Are known and unknown heads disentangled?
Concept Contribution	↑	Do predictions route through the concept module?
Known Concept Alignment	↑	Do concept embeddings point at related tokens?
\bottomrule

Figure 6: Comparison of all-mpnet-base-v2 and Qwen3-Embedding-0.6B embeddings for clustering ∼14M LLM-generated tag annotations via the k-means clustering algorithm. We evaluate Silhouette score (left plot), Davies–Bouldin index (middle plot), and coherence score (right plot) on randomly sampled cluster members. As k increases, all-mpnet-base-v2 consistently yields more coherent clusters than Qwen3-Embedding-0.6B.

Table 6: Compute-optimal scaling exponents and irreducible-loss asymptotes. Subscripts are 90% bootstrap confidence intervals.
\topruleModel	αP	αD	αL	ℒ∞
\midrule Ours (Autoregressive)
AR	0.528[−0.025,+0.023]	0.445[−0.036,+0.114]	−0.060[−0.002,+0.002]	1.857[−0.335,+0.071]
AR+Concept	0.621[−0.041,+0.077]	0.524[−0.112,+0.108]	−0.070[−0.002,+0.003]	1.193[−0.703,+0.394]
\midrule Ours (Causal Diffusion)
CDLM	0.632[−0.091,+0.075]	0.481[−0.111,+0.162]	−0.053[−0.004,+0.004]	2.658[−0.708,+0.181]
CDLM+Concept	0.503[−0.046,+0.051]	0.374[−0.170,+0.174]	−0.072[−0.003,+0.003]	1.942[−1.805,+0.529]
\midrule Masked diffusion (prior)
96	0.514	0.486	—	2.41
146	0.566[−0.022,+0.019]	0.434[−0.019,+0.020]	−0.0496[−0.0004,+0.0003]	—
\midrule Autoregressive (prior)
54	0.490	0.510	—	1.69
124	0.464	0.536	—	—
18	0.524	0.476	—	—
\bottomrule

Figure 7: Stage-2 transformation of noisy LLM tags into a canonical concept: raw tags are embedded and clustered, incoherent clusters are filtered out, each surviving cluster is labeled into a single concept (name with italic description), and semantically adjacent concepts are merged by cosine similarity into one canonical entry.

Table 7: Interpretability scaling fits. Subscripts on e are 90% bootstrap confidence intervals. The 8B columns compare the small-scale extrapolation against the actual Steerling values.
\topruleFamily	β	e [90% CI]	R2	8B pred	8B actual	Δ
\midrule Concept Loss
AR+Concept	0.385	0.002[−0.002,+0.000]	0.704	0.002	—	—
CDLM+Concept	0.054	0.000[−0.000,+0.002]	0.648	0.002	0.002	+0.000
\midrule Concept Independence Loss
AR+Concept	0.279	1.310[−1.310,+0.000]	0.505	1.550	—	—
CDLM+Concept	0.217	0.000[−0.000,+3.687]	0.749	2.157	1.550	−0.611
\midrule Concept Contribution
AR+Concept	0.212	0.937[−0.047,+0.063]	0.630	0.915	—	—
CDLM+Concept	0.134	1.000[−0.149,+0.000]	0.496	0.915	0.876	−0.039
\midrule Known Concept Alignment
AR+Concept	0.099	5.000[−1.190,+0.000]	0.557	4.200	—	—
CDLM+Concept	0.349	3.920[−0.135,+1.080]	0.637	3.870	3.770	−0.100
\bottomrule

Figure 8: UMAP projection of a subsample of concept embeddings across five taxonomy groups. The cards showcase representative concepts from distinct regions of the embedding space to illustrate local semantic clusters.

Table 8: Joint interpretability scaling fits, m⁡(P,D)=e±(APP−α+ADD−β), with α the parameter exponent and β the token exponent. Subscripts on e are 90% bootstrap confidence intervals. The 8B columns compare the small-scale extrapolation against the actual Steerling values.
\topruleFamily	α	β	e [90% CI]	R2	8B pred	8B actual	Δ
\midrule Concept Loss
AR+Concept	0.093	0.767	0.000[−0.000,+0.001]	0.945	—	—	—
CDLM+Concept	0.427	1.255	0.002[−0.000,+0.000]	0.893	0.002	0.002	+0.000
\midrule Concept Independence Loss
AR+Concept	1.085	0.529	1.347[−1.347,+0.745]	0.624	—	—	—
CDLM+Concept	0.904	0.659	0.000[−0.000,+1.908]	0.937	0.624	1.550	+0.921
\midrule Concept Contribution
AR+Concept	0.541	0.739	0.903[−0.028,+0.097]	0.784	—	—	—
CDLM+Concept	0.441	2.077	0.864[−0.012,+0.018]	0.883	0.844	0.876	+0.032
\midrule Known Concept Alignment
AR+Concept	2.660	0.154	4.640[−0.363,+0.360]	0.926	—	—	—
CDLM+Concept	0.022	0.346	5.000[−1.125,+0.000]	0.933	3.967	3.770	−0.194
\bottomrule

Figure 9: Stage 2 concept validation scores. Distributions of average concept-relevance ratings on a 1–5 scale: per-chunk averages (left column) and per-concept averages (right column; each concept’s ratings averaged over the chunks it is assigned to), as rated by the LLM judge over 2.93M chunks (top row) and by human annotators on a 50-chunk sample (bottom row).

Table 9: Steerling-8B pretraining corpus. Token counts are post-tokenization.
\topruleSource	Documents	Chunks	Tokens
\midruleNemotron-CC-HQ (real)	740M	5.1B	547B
Nemotron-CC-HQ (synthetic)	971M	4.8B	498B
peS2o	38.8M	565.4M	59B
arXiv	3.9M	142.2M	20.4B
Wikipedia & Wikibooks	6.1M	36.8M	3.8B
OpenWebMath	2.9M	76.8M	12.1B
Algebraic Stack	2.8M	65.6M	12.1B
StarCoder	78.6M	317M	91.4B
\midruleTotal	1.84B	11.1B	1.24T
\bottomrule

Figure 10: Stage 3 annotator validation on held-out chunks. Distributions of average predicted-concept relevance ratings from the LLM judge: per-chunk averages (left) and per-concept averages (right).

Table 10: Steering results on random concepts of the pretrained Steerling-8B. Steered denotes layer injection.
\topruleMethod	Concept ↑	Quality ↑	Harmonic ↑
\midruleUnsteered	0.033	1.108	0.065
Prompting	0.908	1.588	1.156
Steered	1.072	0.972	1.020
\bottomrule

Figure 11: Schematic of the LCC-derived taxonomy used to organize the Atlas concept library. The 33,732 canonical concepts are mapped onto approximately 2,600 occupied taxonomy nodes. The figure shows the root structure and expands the Science (Q) branch to illustrate how top-level classes decompose into more specific areas such as Mathematics (QA), Physics (QC), and their subdivisions; other branches are collapsed for readability.

Table 11: Data compositions compared in the mid-training ablation. Each arm is a 10B-token run from the final pretraining checkpoint of 1.2T tokens; entries are token counts in billions.
\topruleSource	Math-heavy	Balanced	Code-augmented	Code-only
\midruleNemotron (real)	5.0B	4.7B	5.0B	–
Dolmino Math	5.0B	2.1B	2.0B	–
StarCoder	–	–	1.0B	10.0B
FLAN	–	1.7B	1.0B	–
peS2o	–	0.6B	0.4B	–
Wikipedia & Wikibooks	–	0.7B	0.5B	–
Stack Exchange	–	0.2B	0.2B	–
\midruleTotal	10.0B	10.0B	10.0B	10.0B
\bottomrule

Figure 12: Distribution of over 33,000 concepts across the top-level LCC classes and notable subclasses. Science (Q) accounts for the largest share, followed by Technology (T), Social Sciences (H), and Medicine (R); all root classes are represented to varying degrees. The distribution mirrors the composition of the pretraining corpus rather than a curatorial choice.

Table 12: Downstream performance of the four mid-training compositions, each a 10B-token run from the final pretraining checkpoint, against the base model. Best in each column in bold. HSwag: HellaSwag; HEval: HumanEval; WinoG: WinoGrande.
\topruleComposition	MMLU	GSM8K	ARC-C	HSwag	HEval	MBPP	WinoG	Avg.
\midrulePretrained model	0.298	0.140	0.484	0.673	0.049	0.004	0.596	0.321
\midruleMath-heavy	0.376	0.441	0.492	0.681	0.037	0.012	0.616	0.379
Balanced	0.416	0.328	0.497	0.693	0.037	0.006	0.616	0.371
Code-augmented	0.416	0.328	0.498	0.693	0.055	0.012	0.628	0.376
Code-only	0.303	0.086	0.434	0.630	0.061	0.012	0.583	0.301
\bottomrule

Figure 13: Phase 2 human-evaluation fit scores by candidate type. Human annotators rated, on a 1–5 scale, how well each candidate name fit the same lifted-word evidence for a concept. Points show mean fit scores and horizontal bars show 95% confidence intervals; the dotted vertical line marks the neutral midpoint of the scale. Atlas labels score comparably to or above independently generated human labels, and both are clearly separated from embedding, taxonomy, and filler distractors, indicating that raters were not merely assigning high scores to any plausible label.

Table 13: The final Steerling-8B midtraining mixture of 150B tokens. Dolmino Math is upsampled roughly twofold. Proportions follow the code-augmented composition of \creftab:midtraining-composition.
\topruleSource	Number of Tokens	Ratio (%)
\midruleNemotron (real)	72.79B	48.5
StarCoder	30.75B	20.5
peS2o	21.98B	14.7
Dolmino Math (∼2×)	16.05B	10.7
FLAN	6.38B	4.3
Wikipedia & Wikibooks	1.50B	1.0
Stack Exchange	0.56B	0.4
\midruleTotal	150B	100.0
\bottomrule

Table 14: Masking schedule ablation, each a 10B-token run from the final pretraining checkpoint. Best in each column in bold. HSwag: HellaSwag; HEval: HumanEval; WinoG: WinoGrande.
\topruleMasking	MMLU	GSM8K	ARC-C	HSwag	HEval	MBPP	WinoG	Avg.
\midrulePretrained model	0.298	0.140	0.484	0.673	0.049	0.004	0.596	0.321
50% uniform	0.416	0.328	0.498	0.693	0.055	0.012	0.628	0.376
80% Gaussian	0.280	0.355	0.500	0.686	0.055	0.008	0.628	0.359
\bottomrule

Table 15: Steering ablation: Adding the steering phases improves every metric on the steering benchmark (150), while LM Harness performance stays mostly unchanged.
\toprule	LM Harness ↑	Steering benchmark ↑
\cmidrule(lr)2-6 \cmidrule(lr)7-9	MMLU	GSM8K	ARC-C	HSwag	WinoG	Concept	Quality	Harmonic
\midruleMath-heavy	0.370	0.431	0.499	0.682	0.618	1.208	0.989	1.088
+ steering	0.384	0.415	0.505	0.681	0.611	1.244	1.139	1.189
\bottomrule

Table 16: Steerling-8B before and after mid-training across the LM Harness suite (accuracy, %). HSwag: HellaSwag; HEval: HumanEval; WinoG: WinoGrande.
\toprule	MMLU	GSM8K	ARC-C	HSwag	HEval	MBPP	WinoG	Avg.
\midrulePretrained	29.8	14.0	48.4	67.3	4.9	0.4	59.6	32.1
Mid-trained	46.4	44.4	52.3	70.3	8.5	11.0	64.2	42.4
\bottomrule

Table 17: Interpretability metrics for Steerling-8B before and after mid-training.
\toprule	Pretrained	Mid-trained
\midrule Concept Loss ↓	0.002	0.002
Concept Independence Loss ↓	1.907	1.546
Concept Contribution ↑	0.851	0.876
Known Concept Alignment ↑	3.730	3.770
\bottomrule

Table 18: Steering benchmark scores for the pretrained and mid-trained Steerling-8B checkpoints. Mid-training improves every steering metric.
\topruleCheckpoint	Concept ↑	Quality ↑	Harmonic ↑	Sample harmonic ↑
\midrulePretrained (1.2T)	1.072	0.972	1.020	0.843
Mid-trained (1.35T)	1.247	1.064	1.148	0.963
\bottomrule

Table 19: Steerling-8B base model against open base models of comparable size. HSwag: HellaSwag; WinoG: WinoGrande. Values marked ∗ are taken from the OLMo 3 report (100); all other peer numbers are from the respective model reports.
\topruleModel	HSwag	WinoG	PIQA	MMLU	ARC-C	GSM8K	Math	Avg.
\midruleSteerling-8B	70.3	64.2	75.9	46.4	52.3	44.4	8.0	51.6
\midruleLLaMA2 7B	76.0	72.5	79.1	45.9	46.3	13.1	4.3	48.2
DeepSeek 7B	75.4	70.5	79.2	48.2	48.1	17.4	6.0	49.3
Gemma 1 7B	81.2	72.3	81.2	64.3	53.2	46.4	24.3	60.4
LLaDA 8B	70.5	74.8	73.6	65.9	45.9	70.3	31.4	61.8
LLaMA3 8B	79.1	77.3	80.6	65.4	53.1	48.7	16.0	60.0
OLMo 2 7B	83.8	77.2	80.1∗	63.7	79.8	67.5	19.1∗	67.3
\bottomrule

Table 20: Concept module notation, grouped by role.
\topruleSymbol	Type	Meaning
\midrule Hidden states
h	vector	Transformer hidden state
h¯	vector	Bottlenecked state passed to the LM head
h¯=k^+u^+ε	equation	Concept module decomposition
\midrule Concept module heads
f	function	Known head
g	function	Unknown head
k=σ⁡(f⁡(h))	vector	Per-concept activation probabilities, known
u=σ⁡(g⁡(h))	vector	Per-concept activation probabilities, unknown
k\textknown	scalar	Top-k count for the known head
k\textunknown	scalar	Top-k count for the unknown head
\midrule Concept embeddings
K	matrix	Known concept embedding matrix
U	matrix	Unknown concept embedding matrix
Ki	vector	Embedding of known concept i
Uj	vector	Embedding of unknown concept j
n	scalar	Number of known concepts
m	scalar	Number of unknown concepts (m≫n)
R	scalar	Factorization rank of unknown embedding matrix
\midrule Concept contributions
k^=∑ikiKi	vector	Known concept contribution
u^=∑jujUj	vector	Unknown concept contribution
ε=h−k^−u^	vector	Residual term
\midrule Logit decomposition
Wy	vector	Row of the LM head for output token y
ℓy	scalar	Output logit for token y
\bottomrule

Table 21: Concept module notation, grouped by role.
\topruleSymbol	Type	Meaning
\midrule Losses
ℒ\textLM	loss	Language modeling loss (ℒ\textMDM on h¯)
ℒ\textconcept	loss	Concept loss (chunk-level BCE)
ℒ\textrec	loss	Reconstruction loss for the unknown head
ℒ\textindep	loss	Independence loss between k^ and u^
ℒ	loss	Combined training objective
λ\textconcept,λ\textrec,λ\textindep	scalars	Loss weights
\midrule Supervision and targets
yc	scalar	Ground-truth chunk label, known concept c
kc\textchunk	scalar	OR-aggregated chunk-level activation
ki\textGT	scalar	Ground-truth activation of known concept i
k^\textGT	vector	Ground-truth known concept contribution
u^\textGT=h−k^\textGT	vector	Target for the unknown head
\midrule Independence loss
Hk,Hu	matrices	Stacked per-token k^, u^ over a minibatch
𝝁k^,𝝁u^	vectors	Column means of Hk, Hu
Φ	matrix	Centered known features
Ψ	matrix	Centered unknown features
\midrule Training dynamics
ℳ	set	Masked token positions in the minibatch
tb	scalar	Per-block noise level (block b)
α\textknown(s)	scalar	Teacher forcing prob., known, step s
α\textunknown(s)	scalar	Teacher forcing prob., unknown, step s
p\textcfg	scalar	Dropout rate for the known head
pε	scalar	Dropout rate for the residual ε
B	scalar	Minibatch size
b	scalar	Block size in causal block-diffusion
\bottomrule

Table 22: Robustness checks for the Phase 2 comparison between pipeline labels and independently generated human labels.
\topruleAnalysis	Result
\midruleMean Phase 2 fit score	LLM 3.98 vs. human 3.50
Top-two rating rate (≥4)	LLM 79% vs. human 63%
Bayesian cumulative-link model	OR 2.38, 95% CrI [1.23,4.01]
Posterior probability of LLM advantage	0.99
Paired comparison probability	0.62, bootstrap 95% CI [0.58,0.66]
Gaussian mixed model	+0.47 points, 95% CI [0.28,0.65]
\bottomrule

Table 23: Notation introduced in the scaling-law analysis, grouped by role: compute and resources, validation losses, power-law parameters, and interpretability scaling.
\topruleSymbol	Type	Meaning
\midrule Compute and resources
C	scalar	Total training FLOPs
M	scalar	Per-token FLOPs (forward + backward)
P	scalar	Non-embedding parameter count
D	scalar	Number of training tokens (total)
Di	scalar	Training tokens for the i-th checkpoint
\midrule Validation losses
ℒi	scalar	Measured validation loss for checkpoint i
ℒ∗(C)	function	Compute-optimal validation loss at budget C
ℒ∞	scalar	Irreducible validation loss
ℒ⁡(P,D)	function	Joint Chinchilla loss surface
ℒ\textfit	function	Huber loss minimized in the joint fit
\midrule Power-law parameters
P∗(C)	function	Compute-optimal parameter count at budget C
aP,aL	scalars	Power-law coefficients (parameter and loss)
αP	scalar	Exponent on compute for parameter scaling
αL	scalar	Exponent on compute for loss scaling
αD	scalar	Exponent on compute for training-token scaling
AP,AD	scalars	Chinchilla coefficients (parameter and data sides)
α,β	scalars	Chinchilla exponents (parameter and data sides)
\midrule Interpretability scaling
m⁡(C)	function	A metric as a function of compute
e	scalar	Irreducible value of a metric
A,β	scalars	Coefficient and exponent for metric scaling
\bottomrule

Table 24: Backbone architectures used across all four families. Backbone parameter counts exclude embeddings; +Concept totals include the concept module heads (concept classifier, and factorized unknown head concept embeddings are excluded). Sequence length is 4096 throughout.
\topruleSize	Layers L	Hidden d	Backbone params	+Concept total params
\midrule10M	6	320	9.2M	82.8M
25M	6	512	23.6M	110.2M
85M	10	768	86.5M	190.5M
200M	13	1024	197.7M	319.1M
400M	17	1280	401.1M	540.0M
800M	17	1792	779.9M	953.6M
1.5B	20	2304	1,510M	1,718M
3B	24	3072	3,228M	3,489M
5B	24	3840	5,694M	6,007M
\bottomrule

Table 25: IsoFLOP target compute budgets per family.
\topruleFamily	Slice 1	Slice 2	Slice 3	Slice 4
\midruleAR	6×1018	1019	3×1019	1020
CDLM	6×1018	1019	3×1019	1020
AR+Concept	1019	3×1019	1.1×1020	3.09×1020
CDLM+Concept	1019	3×1019	1.1×1020	3.09×1020
\bottomrule

研究结果

在跨越三个数量级计算量的IsoFLOP规模扫描实验中,加入概念模块只使计算最优的缩放指数产生一个很小且固定的偏移,这一可解释性代价并不随规模增大而增长。
同一组实验显示,随着计算量增加,概念预测准确率、已知与未知表示的分离程度、预测经由概念路径而非残差路径传递的比例、以及概念嵌入与人类标签的对齐程度等可解释性指标均有所提升。
利用联合Chinchilla形式的规模法则从小规模实验外推,预测出的验证损失与Steerling-8B实际值相差仅0.11 nats以内,四项可解释性指标中有三项也落在很窄的预测区间内。
在1.2万亿token预训练加1500亿token中期训练后,Steerling-8B与使用约2至16倍计算量训练的同等参数规模公开同类模型相比,平均基准表现差距在约10%以内。
在Atlas流水线的标签与概念验证中,LLM评审者和人类标注者对标签、概念相关性的平均评分总体较高,且Atlas生成的概念标签评分与人类独立撰写的标签相当或更高,并明显区别于嵌入、分类体系和干扰项标签。

可应用场景

可以把追踪输出所依赖的输入token、概念和训练样本用作诊断模型行为的审计工具。
无需重新训练即可通过概念操控在推理时把生成内容向某个概念方向增强或抑制,可用作一种控制生成的手段。
训练数据归因功能可以尝试用于对齐微调、事实来源追溯,或检查模型依赖的是专有微调数据还是预训练知识。
Atlas这种三阶段(打标签、聚类、训练标注模型)流水线可为其他需要在网络规模语料上做概念标注的研究提供参考模板。

局限与待验证事项

训练数据归因只是在表示空间中做近似相似度检索,并非因果影响估计,因此不能证明某条训练数据真的导致了某个输出。
三万三千七百三十二个概念的概念库反映的是这份特定预训练语料的构成(科学、技术、社会科学、医学等),尚未验证是否能推广到领域或语言构成差异很大的语料上。
Steerling-8B使用远少于同类模型2至16倍的计算量,平均性能差距约在10%以内,但在个别基准上可能仍存在差距。
概念模块的参数占比(8B模型下为4%,在相同参数化假设下预计在前沿规模会降到1%以下)是外推估计,并未在真正的前沿规模模型上直接测得。
人类评估样本规模有限(如50个文本块样本、1025条独立评分),仍需更大范围的人工验证。

为什么重要

如果这一结论成立,它挑战了可解释性与模型能力必然此消彼长的传统假设,为在训练阶段就把可审计、可纠正、可控制的能力内建到大模型中提供了一条路径,而不必等训练完成后再去反推解释。这可能转化为实用的诊断工具,用于追踪模型行为源头、定位相关训练数据,并在不重新训练的情况下通过概念操控修正模型行为。

本文术语

概念瓶颈(concept bottleneck) · 在模型内部表示和最终输出之间插入的一层人类可理解的'概念'单元,使输出可以分解为这些概念的贡献之和
扩散语言模型(diffusion language model) · 通过对文本进行掩码再学习还原的方式训练的语言模型,与逐词从左到右预测的自回归模型不同
事后可解释性(post-hoc interpretability) · 在模型训练完成之后,用外部分析工具去解释其行为的传统做法
IsoFLOP规模扫描 · 在固定总计算预算下,改变模型大小和数据量来观察性能变化趋势的实验方法
积分梯度(Integrated Gradients) · 通过沿着从基线输入(此处为[MASK]标记)到实际输入的路径对梯度做积分,来计算每个输入token重要性的方法

论文原文摘要（英文）

Interpretability is often treated as a tax on capability: language models are trained as opaque systems, then explained after the fact, with methods whose reliability is difficult to establish. In this work, we challenge this premise. Rather than reverse-engineering a model, we make interpretability a constraint of the training pipeline, optimized alongside the language modeling objective. Across three orders of magnitude of compute, on both autoregressive and diffusion language models, interpretability scales with capability rather than against it. Surprisingly, model representations become more disentangled and aligned with human-understandable concepts with scale. We instantiate the training-time recipe with Steerling-8B, a diffusion language model with a causal attention mask. For any group of generated tokens, Steerling-8B attributes the output to relevant input tokens, human-understandable concepts, and training data. This enables closed-loop intervention: diagnose an output through its concept or feature attribution, retrieve similar training data, and correct the behavior through concept steering without retraining. Steerling-8B remains competitive with open peer models trained on substantially 2-16x more compute, suggesting a different scaling paradigm: interpretability can be designed into training, and it improves with scale.

作者 · Guide Labs Team

在 arXiv 阅读

METAL MEDIA 最新报道

图片来源： Guide Labs Team et al., arXiv:2608.07594, CC BY 4.0