AI가 지어낸 문장을 한 단어씩 보지 말고 흐름으로 봐야 더 잘 잡힌다

arXiv:2608.181152026-08-20

Temporal Multi-Signal Fusion for Token-Level Hallucination Detection

AI 언어모델이 지어낸 말(할루시네이션)을 찾아내는 기존 방법들은 단어 하나하나를 따로 채점해서, 모델이 자신 있게 틀린 말을 할 때 놓치기 쉽다. 이 논문은 할루시네이션이 한 단어가 아니라 여러 단어에 걸쳐 이어지는 구간이라는 점에 착안해, 33가지 신호를 하나로 묶어 문장의 앞뒤 흐름까지 함께 보는 모델(BiGRU)로 판별했다. 그 결과 기존 방식보다 정확도(AUC)가 11점 높은 0.840을 기록했고, 이 향상분의 44%는 순서(시간적 맥락)를 본 덕분이었다.

METAL MEDIA 해설 도표

AI가 지어낸 문장을 한 단어씩 보지 말고 흐름으로 봐야 더 잘 잡힌다

01기존 할루시네이션 탐지기는 단어 하나씩 독립적으로 점수를 매겨, 모델이 확신을 갖고 틀리는 경우를 놓쳤다.
02텍스트 통계, 문장 간 모순 여부를 보는 NLI, 언어모델이 느끼는 놀라움 정도(서프라이절) 등 33가지 신호를 한 단어마다 뽑아 앞뒤 흐름을 함께 읽는 양방향 순환신경망(BiGRU)에 넣었다.
03모델 내부 정보 없이 생성된 텍스트와 외부 신호만으로 RAGTruth 벤치마크에서 AUC 0.840을 달성해, 단어를 독립적으로 보는 로지스틱회귀 대비 11점 높았다(10회 반복 실험, p=0.002).
04성능 향상의 요인을 뜯어보니 순서(시간 맥락)가 44%, 시퀀스 전체를 보는 것이 24%, 모델의 비선형 표현력이 32%를 차지했다.
05학습에 쓰지 않은 다른 언어모델이 만든 글에도 적용했을 때 정확도 하락이 4% 미만이었고, 조건부무작위장(CRF) 모델은 점수 계산 방식을 forward-backward 방식으로 바꾸자 최대 17.9점까지 성능이 올라갔다.

METAL MEDIA이 원문을 바탕으로 재구성한 해설 도표이며, 논문 저자의 원문 figure가 아닙니다.

무엇을 했나

기존 할루시네이션 탐지기는 단어 하나씩 독립적으로 점수를 매겨, 모델이 확신을 갖고 틀리는 경우를 놓쳤다.
텍스트 통계, 문장 간 모순 여부를 보는 NLI, 언어모델이 느끼는 놀라움 정도(서프라이절) 등 33가지 신호를 한 단어마다 뽑아 앞뒤 흐름을 함께 읽는 양방향 순환신경망(BiGRU)에 넣었다.
모델 내부 정보 없이 생성된 텍스트와 외부 신호만으로 RAGTruth 벤치마크에서 AUC 0.840을 달성해, 단어를 독립적으로 보는 로지스틱회귀 대비 11점 높았다(10회 반복 실험, p=0.002).
성능 향상의 요인을 뜯어보니 순서(시간 맥락)가 44%, 시퀀스 전체를 보는 것이 24%, 모델의 비선형 표현력이 32%를 차지했다.
학습에 쓰지 않은 다른 언어모델이 만든 글에도 적용했을 때 정확도 하락이 4% 미만이었고, 조건부무작위장(CRF) 모델은 점수 계산 방식을 forward-backward 방식으로 바꾸자 최대 17.9점까지 성능이 올라갔다.

Table 1: Positioning on RAGTruth. †Span F1. ‡10 seeds. §0.73 = the published Lookback Lens (LogReg) classifier; 0.838 = the same attention features under our BiGRU sequence model. Both use a Qwen3-14B proxy on the open-LLM subset; see §5.9 for the full fair comparison.
Method	Access	Metric	Score
ReDeEP	Activations	AUC	0.733
RagtStacking	Activations	AUC	0.836
Lookback Lens§	Attention	AUC	0.73 / 0.838
LettuceDetect	Black-box	Span F1†	0.589
RL4HS	Black-box	Span F1†	0.583
HaluGate	Black-box	Token F1	0.590
Ours (BiGRU)	Black-box	AUC‡	0.840±.007

Table 2: Per-token feature groups.
Group	Signals	Dim
Text	Surface, overlap, running statistics	20
NLI	Entailment scores + temporal derivatives	7
LM	Surprisal, rank + fallback handling	6
	Total	33

Table 3: Sequence labeling architectures. All models use h=64. CRF variants use forward-backward marginals for ranking (Section 5.10).
Model	Temporal scope	Params
LogReg	None (per-token)	34
MLP	None (per-token)	112K
1D-CNN	Local (kernel ≤7)	61K
BiGRU	Full sequence	121K
BiLSTM	Full sequence	121K
Transformer	Full sequence (attention)	106K
BiGRU-CRF	Full + transitions	121K
BiLSTM-CRF	Full + transitions	121K

Table 4: Research questions and corresponding experiments.
RQ	Experiments
RQ1: Temporal	Main ablation (Section 5.1),
dependencies	decomposition (5.3),
	directional ablation (5.4)
RQ2: Multi-signal	Signal ablation (Section 5.1),
fusion	pointwise baselines (5.5),
	Lookback comparison (5.9)
RQ3: General-	Cross-model (5.7),
ization	cross-dataset (5.8)

Table 5: Token-level AUC and F1 on RAGTruth test set. CRF models use softmax scoring; see Table 12 for forward-backward results.
Model	Text Only	All Signals
AUC	F1	AUC	F1
LogReg	0.702	0.133	0.730	0.141
1D-CNN	0.788	0.208	0.807	0.216
BiLSTM	0.826	0.227	0.843	0.247
BiGRU	0.832	0.232	0.845	0.242
Transformer	0.802	0.233	0.804	0.229
BiGRU-CRF	0.806	0.196	0.630	0.245
BiLSTM-CRF	0.603	0.188	0.781	0.164
Trans-CRF	0.668	0.235	0.691	0.227

Table 6: Mean ± std across 10 seeds (all signals). MLP is a capacity-matched token-independent baseline (112K parameters ≈ BiGRU’s 121K).
Model	AUC	F1	AP
LogReg	.730±.001	.141±.000	.093±.001
MLP	.765±.002	.176±.003	.127±.002
1D-CNN	.813±.002	.218±.004	.197±.005
BiLSTM	.835±.008	.242±.008	.270±.017
BiGRU	.840±.007	.257±.010	.282±.018
Transformer	.819±.008	.246±.009	.234±.012

Table 7: Controlled decomposition of BiGRU’s advantage over LogReg (10 seeds, all signals).
Model	Adds	AUC	Δ
LogReg	Per-token, linear	.730	–
MLP (112K)	Nonlinear capacity	.765	+3.5 (32%)
Shuffled BiGRU	Seq. aggregation	.791	+2.6 (24%)
BiGRU	Temporal order	.840	+4.9 (44%)

Table 8: Directional ablation (10 seeds, all signals).
Model	AUC	Avg Precision
ForwardGRU	.802±.008	.217±.014
BackwardGRU	.817±.005	.223±.015
BiGRU	.840±.007	.282±.018

Table 9: Individual features as standalone detectors.
Signal	Method class	AUC	AP
LM entropy	Token entropy	.551	.047
LM log-prob	Perplexity	.549	.046
NLI contradiction	NLI-based	.639	.087
NLI entailment drop	Entailment check	.641	.066
Context overlap	Lexical overlap	.597	.054
Running novelty	Novelty tracking	.616	.055
LogReg, all 33 features	.730	.103
BiGRU, all 33 features	.840	.282

Table 10: Cross-dataset transfer (mean ± std, 10 seeds, all signals).
Model	R → P	P → R
AUC	F1	AUC	F1
LogReg	.618	.712	.692	.161
BiGRU	.634	.709	.744	.185
Transformer	.624	.719	.703	.144

Table 11: Fair comparison on the open-LLM subset of RAGTruth (5 seeds). Combined = our 33-dim + Lookback features.
Features	Model	AUC	AP
Proxy: Qwen3-14B
Lookback only	BiGRU	.838±.015	.368
Combined (193-dim)	BiGRU	.866±.007	.425
No attention access
Ours only (33-dim)	BiGRU	.819±.013	.298

Table 12: CRF scoring: softmax vs. forward-backward (FB) marginals. Softmax values differ from Table 5 because CRF softmax scores are unstable across seeds (see text).
Model	Softmax	FB	Δ
BiGRU-CRF	0.666	0.845	+0.179
BiLSTM-CRF	0.785	0.807	+0.022
Trans-CRF	0.741	0.818	+0.077

Table 13: Complete list of 33 per-token features. The “Context” column indicates whether the feature encodes temporal information: point = depends only on the current token; cumul = cumulative statistic up to position t; window = sliding window average/max; delta = finite difference between adjacent positions; sent = sentence-level (projected to all tokens in the sentence).
#	Group	Feature	Context
1	Text	Word length (/20)	point
2	Text	Is numeric	point
3	Text	Is capitalized	point
4	Text	Absolute position t	point
5	Text	Relative position t/(T−1)	point
6	Text	Unigram overlap 𝟏[wt∈C]	point
7	Text	Bigram overlap	point
8	Text	Trigram overlap	point
9	Text	Entity indicator	point
10	Text	Cumulative overlap ratio	cumul
11	Text	Running novelty rate	cumul
12	Text	Consecutive novel count	cumul
13	Text	Windowed novelty (5 tok)	window
14	Text	Windowed novelty (10 tok)	window
15	Text	Windowed novelty (20 tok)	window
16	Text	Novelty velocity (1st diff)	delta
17	Text	Novelty accel. (2nd diff)	delta
18	Text	Sentence position index	point
19	Text	Sentence-relative position	point
20	Text	Running mean word length	cumul
21	NLI	P(contradiction∣C,s)	sent
22	NLI	P(entailment∣C,s)	sent
23	NLI	P(neutral∣C,s)	sent
24	NLI	Running mean contradiction	cumul
25	NLI	Contradiction delta	delta
26	NLI	Windowed max contr. (10 tok)	window
27	NLI	Entailment drop	sent
28	LM	Log-probability (sum subwords)	point
29	LM	Next-token entropy	point
30	LM	Mean subword rank (log)	point

Table 14: Token AUC by sequence length quartile. The BiGRU–LogReg gap grows from +8.1 (Q1) to +12.2 (Q4) as longer sequences provide more temporal context.
Quartile	Mean Len.	LogReg	BiGRU	Δ
Q1 (short)	62	0.781	0.863	+0.081
Q2	103	0.726	0.814	+0.087
Q3	135	0.708	0.824	+0.116
Q4 (long)	203	0.730	0.851	+0.122

Table 15: Onset detection metrics (all signals, tolerance k=3). 1D-CNN achieves the best F1 due to high recall; BiGRU has the best precision.
Model	Prec@3	Rec@3	F1@3	Early
LogReg	.100	.517	.155	.423
1D-CNN	.147	.520	.208	.412
BiGRU	.171	.298	.196	.279

Table 16: BiGRU hyperparameter sweep (mean ± std AUC, 10 seeds). Performance plateaus at h=64 and degrades for h≥128.
h	Layers	AUC	Params
32	1	.837±.005	15K
32	2	.842±.004	34K
32	3	.840±.007	53K
64	1	.841±.004	46K
64	2	.839±.005	121K
64	3	.843±.005	195K
128	1	.836±.006	158K
128	2	.832±.009	455K
128	3	.830±.011	751K
256	1	.831±.014	579K
256	2	.816±.006	1.8M
256	3	.826±.011	2.9M

Table 17: Training recipe ablation on BiGRU; the first row (BCE + constant) is the baseline. No recipe significantly outperforms it, suggesting the AUC ceiling is determined by the features rather than the optimization.
Recipe	AUC	F1	P	R
BCE + constant	.839±.005	.254	.158	.654
BCE + plateau	.837±.006	.249	.154	.661
BCE + cosine	.840±.006	.248	.154	.661
Focal + constant	.840±.006	.248	.152	.670
Focal + plateau	.839±.007	.249	.153	.663
Focal + cosine	.839±.007	.242	.148	.675

Table 18: Training recipe ablation on Transformer Pre-LN (10 seeds each).
Recipe	AUC	F1
BCE + constant	.805±.008	.204
BCE + plateau	.803±.011	.203
BCE + cosine	.796±.008	.198
Focal + constant	.799±.008	.209
Focal + plateau	.801±.010	.206
Focal + cosine	.797±.009	.201

Table 19: Full Lookback Lens comparison (open-LLM subset, 5 seeds).
Features	Model	AUC	F1	AP
TinyLlama-1.1B proxy
Lookback	LogReg	.699	.194	.173
Lookback	BiGRU	.822	.355	.344
Combined	BiGRU	.853	.355	.401
Qwen3-14B proxy
Lookback	LogReg	.726	.204	.187
Lookback	BiGRU	.838	.384	.368
Combined	BiGRU	.866	.391	.425
Ours only	BiGRU	.819	.297	.298

Table 20: Architecture comparison with cosine schedule (literature recipes, 5–10 seeds). BiGRU, Mamba, and BiGRU+Attention converge to ∼0.84.
Architecture	AUC	F1	Params	Seeds
BiGRU	.840±.007	.257	121K	10
BiGRU+Attention	.840±.005	.251	113K	10
Mamba	.838±.006	.244	203K	9
BiLSTM	.835±.008	.242	121K	10
BixLSTM	.827±.006	.233	125K	10
DilatedCNN	.826±.006	.244	61K	10
Transformer	.796±.008	.204	106K	10

Table 21: Same architectures with ReduceLROnPlateau + early stopping (max 50 epochs, patience 10, 5 seeds). Plateau scheduling raises the ceiling from 0.840 to 0.845 and eliminates most architecture differences.
Architecture	AUC	Δ	Best Ep	Params
BiLSTM	.845±.003	+.010	17	121K
Mamba	.844±.005	+.006	20	203K
BiGRU+Attn	.844±.002	+.004	12	113K
BiGRU	.843±.003	+.003	8	121K
BixLSTM	.843±.004	+.016	19	125K
DilatedCNN	.833±.002	+.007	13	61K
Transformer	.827±.002	+.031	46	106K

Table 22: Probing comparison: hidden states from the generating model vs. our 33-dim black-box features. Each generator is evaluated only on its own outputs. Hidden states are extracted from 4 evenly spaced layers (dim=16,384).
	LLaMA-2-7B	Mistral-7B
Method	AUC	F1	AUC	F1
Linear probe on HS	.543	.118	.574	.142
BiGRU on HS	.637	.195	.631	.201
BiGRU on 33-dim (ours)	.770	.245	.833	.355
BiGRU on 33-dim + HS	.748	.215	.793	.282

Table 23: Per-task AUC for five architectures (3 seeds, plateau+ES). QA is easiest across all architectures; Data2txt shows the largest temporal advantage.
Architecture	QA	Data2txt	Summary
Mamba	.890±.007	.827±.009	.766±.005
BiGRU	.885±.002	.833±.006	.758±.017
BiGRU+Attn	.885±.004	.830±.008	.770±.010
BixLSTM	.884±.003	.815±.017	.771±.013
DilatedCNN	.883±.005	.795±.009	.760±.008

왜 중요한가

할루시네이션은 확산되기 시작하면 이어지는 단어들도 계속 틀리게 만드는데, 이 흐름을 포착하면 폐쇄형(비공개) AI 모델의 출력물도 내부 정보 없이 검증할 수 있다는 뜻이다. RAG(검색증강생성) 시스템처럼 의료·법률·금융 분야에 AI 답변을 쓰는 서비스에서 잘못된 문장을 더 정확히 걸러낼 수 있는 실용적 방법이 된다.

이 논문의 용어

할루시네이션(hallucination) · AI 언어모델이 사실이 아닌 내용을 그럴듯하게 지어내는 현상
NLI(자연어추론) · 두 문장이 서로 함의·모순·중립 관계인지 판단하는 기술
서프라이절(surprisal) · 언어모델이 특정 단어를 얼마나 예상 밖으로 느끼는지를 수치화한 값
BiGRU · 문장을 앞에서 뒤로, 뒤에서 앞으로 동시에 읽어 맥락을 파악하는 양방향 순환신경망
AUC · 모델이 정답과 오답을 얼마나 잘 구별하는지 나타내는 지표, 1에 가까울수록 좋음
조건부무작위장(CRF) · 인접한 단어의 라벨이 서로 영향을 주도록 모델링하는 구조화된 예측 기법

본문에 싣지 못한 그림

Figure 2: Decomposition of BiGRU’s 11-point advantage over LogReg. Temporal order is the largest contributor (44%).
Figure 3: Feature dynamics aligned to hallucination span onset (1,136 spans). LM entropy spikes at onset; context overlap drops; NLI contradiction rises gradually; BiGRU probability accumulates monotonically.
Figure 4: AUC distribution across 10 seeds for key architectures. Temporal models (BiGRU, BiLSTM) consistently outperform non-temporal baselines (LogReg, MLP).
Figure 5: Token AUC across all architecture–signal combinations (single seed). Non-CRF temporal models (BiGRU, BiLSTM) consistently outperform baselines across all signal configurations.
Figure 6: Leave-one-out cross-model generalization (all signals, 10 seeds). GPT-4 is the hardest transfer target; LLaMA-2-70B is the easiest.
Figure 7: Waterfall decomposition of signal contributions to BiGRU’s AUC. NLI and LM features provide complementary gains beyond text features alone.
Figure 8: Precision-recall curves for all non-CRF architectures (all signals, seed 42). The low overall precision reflects RAGTruth’s 5.6% hallucination rate.
Figure 9: Per-task AUC comparison (all signals). The temporal advantage of BiGRU over LogReg is largest for Data2txt.
Figure 10: Token AUC by sequence length quartile. The BiGRU–LogReg gap grows from +8.1 (Q1) to +12.2 (Q4).
Figure 11: CRF scoring methods compared. Forward-backward marginals recover up to +17.9 AUC points over softmax scoring.
Figure 12: Natural counterfactual: same query, different LLMs, one hallucinating. NLI contradiction separates the two; LM entropy does not; BiGRU integrates both.
Figure 13: Bidirectional dataset transfer. Reverse transfer (PsiloQA to RAGTruth) outperforms forward despite fewer training examples.
Figure 14: Architecture comparison (cosine schedule). All temporal architectures with full-sequence scope converge to ∼0.84 AUC. With plateau scheduling (Table 21), the ceiling rises to 0.845.

원문에서 그림 보기 →

저자 · Igor Itkin

arXiv에서 원문 보기

AI가 지어낸 문장을 한 단어씩 보지 말고 흐름으로 봐야 더 잘 잡힌다

AI가 지어낸 문장을 한 단어씩 보지 말고 흐름으로 봐야 더 잘 잡힌다

무엇을 했나

왜 중요한가

이 논문의 용어

본문에 싣지 못한 그림

최신 논문

METAL MEDIA 최신 기사