InSight-doc: Agentic Visual Perception for Long-Document Understanding
AI가 문서를 훑어보다가 필요한 부분만 확대해서 읽으면, 더 정확하고 더 빠르게 답한다
InSight-doc은 긴 문서를 다룰 때 처음엔 저해상도로 전체를 훑어보고, 답을 찾기 위해 필요한 영역만 스스로 확대(줌인)해서 확인하는 AI 에이전트다. 외부 검색 도구 없이 모델 자신이 어디를 더 자세히 봐야 할지 판단하고 크롭 이미지를 추론 과정에 이어붙인다. 8B 크기 모델로 학습시킨 결과, 문서 질의응답 정확도가 기존 대비 4.3~16.4점 오르고, 긴 문서에서는 헛소리(근거 없는 답변) 비율이 40% 이상 줄고 추론 속도도 41~68% 빨라졌다.
METAL MEDIA 해설 도표
InSight-doc의 저해상도→줌인 추론 흐름
증거 상태측정 결과가 보고됨
저해상도 전체 문서 입력문서 전체 페이지를 낮은 해상도(예: 50DPI)로 축소해 모델에 처음 제공, 토큰 수를 크게 줄인다.
생각과 줌인 도구 호출모델이 <think>로 추론하며 특정 페이지·영역을 지정해 <tool_call>로 확대를 요청한다.
고해상도 영역 크롭 삽입요청된 영역을 원본 고해상도 이미지에서 잘라내 추론 과정에 새로운 시각 증거로 덧붙인다.
반복적 증거 수집필요하면 여러 차례 줌인을 반복해 서로 다른 페이지·영역의 증거를 모은다.
최종 답변 생성충분한 증거가 모이면 <answer>로 최종 답을 내거나, 근거가 없으면 답할 수 없다고 표시한다.
METAL MEDIA이 원문을 바탕으로 재구성한 해설 도표이며, 논문 저자의 원문 figure가 아닙니다.
무엇을 했나
문서 이해 AI는 페이지를 고해상도로 전부 넣으면 느리고, 저해상도로 넣으면 세부 정보를 놓치는 딜레마가 있다. InSight-doc은 저해상도로 시작해 필요한 부분만 원본 고해상도에서 잘라내는 방식으로 이 딜레마를 해결한다.
학습을 위해 질문-답변 데이터에서 문서 없이도 풀리는 문제, 저해상도로도 이미 풀리는 문제를 걸러내고, 남은 어려운 문제에 대해 두 개의 AI 에이전트(reasoner와 searcher)가 협력해 어디를 확대해야 하는지 보여주는 시범 데이터(1만7900여 개)를 만들었다. 이 시범을 따라 배우는 지도학습(SFT)과, 정답 여부만으로 보상을 주는 강화학습(RL, 1만9200여 개 어려운 문제)을 순서대로 적용했다.
DUDE, MP-DocVQA 같은 표준 문서 질의응답과 MMLongBench-Doc, LongDocURL 같은 긴 문서(평균 49~86쪽) 벤치마크에서, 저해상도 입력 기준 평균 정확도가 기반 모델(Qwen3-VL-8B) 대비 16.4점, 중간 해상도에서는 4.3점 높아졌다.
답할 수 없는 질문(문서에 근거가 없는 질문)에 대해 함부로 답을 지어내지 않는 능력도 크게 개선되어, F1 점수가 기반 모델보다 최대 25.9점 올랐다.
같은 정확도를 유지하면서도 토큰 수를 최대 66~69%, 지연 시간을 최대 71%까지 줄여, 정확도와 효율성을 동시에 개선하는 결과를 보였다.
Figure 1: InSight-doc substantially reduces hallucination, sequence length, and latency while improving accuracy on long-document VQA. Hallucination rate is measured on unanswerable questions as the rate of non-abstaining answers. More details in Appendix F.1.
Table 1: Relative-latency upper bound for representative parameters. Here, n is the number of zoom-in tool calls.
r
n=0
n=1
n=2
n=3
n=4
0.25
0.046
0.124
0.238
0.389
0.576
0.35
0.090
0.189
0.325
0.498
0.707
0.50
0.190
0.336
0.519
0.738
0.994
Figure 2: Illustration of InSight-doc on a long-document VQA example. Starting from a low-resolution overview, InSight-doc performs an interleaved multimodal chain-of-thought: each round emits a thought (<think>) and a zoom-in tool call (<tool_call>) specifying img_idx, label, and bbox; the cropped high-resolution region is then appended as visual evidence. After multiple rounds of active perception, the model produces the final <answer>. Images shown above are not in the exact scales seen by the model. More examples can be found in Appendix G.
Table 2: Comparison with frontier models on medium-to-long document VQA. The models are compared under two initial resolution settings: low (r=0.25, DPI 50) and medium (r=0.5, DPI 100). The inference mode “E2E” means single-turn QA without tool use, while “Agent” allows multi-turn zoom-in tool use. LLM-as-judge accuracy is reported on all benchmarks. Input page limit is set to 40 for all models due to API limit on some closed models. The limit only affects MMLongBench-Doc and LongDocURL. The unlimited results can be found in Appendix F.2.
DUDE
MP-DVQA
MMLong.
LongDoc.
Average
0.25
0.5
0.25
0.5
0.25
0.5
0.25
0.5
0.25
0.5
Closed proprietary models
GPT-5.4-nano
E2E
52.8
66.0
64.2
83.4
33.6
52.2
54.4
72.7
51.2
68.6
GPT-5.4-mini
E2E
63.2
71.6
78.3
88.9
46.4
58.8
69.3
77.6
64.3
74.2
GPT-5-mini
E2E
63.9
70.0
81.3
89.5
48.0
57.2
70.3
80.4
65.9
74.3
Gemini-3.1-Flash-lite
E2E
70.2
70.8
87.5
89.4
57.4
58.3
75.8
75.7
72.7
73.5
Gemini-3-Flash
E2E
72.0
72.4
89.3
89.9
62.4
61.8
77.6
76.3
75.3
75.1
Open models
InternVL3-8B
E2E
52.7
62.2
66.5
84.7
13.6
33.5
26.7
42.9
39.9
55.8
GLM-4.6V-Flash (9B)
E2E
40.7
53.4
50.9
72.5
14.8
15.0
24.6
26.0
32.8
41.7
Qwen3-VL-8B
E2E
52.9
68.5
65.1
84.9
33.7
51.4
50.5
68.4
50.5
68.3
Qwen3-VL-8B (w/ zoom)
Agent
55.1
66.4
66.1
82.6
33.2
48.8
47.1
63.8
50.4
65.4
InSight-doc models
InSight-doc-8B (SFT)
Agent
60.8
67.2
72.2
79.5
36.5
48.0
57.0
63.7
56.6
64.6
InSight-doc-8B (SFT+RL)
Agent
70.1
73.8
83.4
87.6
50.8
58.6
63.3
70.5
66.9
72.6
Δ w.r.t. Qwen3-VL-8B
+17.2
+05.3
+18.3
+02.7
+17.1
+07.2
+12.8
+02.1
+16.4
+04.3
Figure 3: Overview of data construction pipeline. The top panel shows a three-stage filtering and CoT construction process: Stage 1 discards questions answerable without the document, Stage 2 discards questions already answerable at low DPI without zoom, and Stage 3 uses InSight-o3 to construct zoom-in CoTs for the remaining items, routing successes to SFT and failures to RL. The bottom panel illustrates the InSight-o3 two-agent trajectory, where the vReasoner and vSearcher iteratively produce reasoning steps, zoom-in requests, bounding boxes, and crops, which are then merged into a single flat multimodal CoT used as the imitation target for InSight-doc.
Table 3: Comparison with frontier models on general high-resolution VQA. LLM-as-judge accuracy is reported under two initial resolution settings: r=0.25 and r=0.5. The inference modes follow Table 2.
MME-RWℓ
O3-Bench
0.25
0.5
0.25
0.5
GPT-5.4-nano
40.3
47.9
29.6
32.8
GPT-5.4-mini
47.4
57.2
40.9
56.2
Gemini-3.1-Flash-lite
47.8
51.8
44.6
51.0
Qwen3-VL-8B
41.0
49.2
22.3
35.1
Qwen3-VL-8B (w/ zoom)
42.5
50.6
20.3
35.4
InSight-doc-8B (SFT+RL)
48.2
52.9
24.1
43.8
Figure 4: Accuracy vs. efficiency on four multi-page document VQA benchmarks. We compare InSight-doc (ours, green) with Qwen3-VL-8B without zoom-in calls (purple) and with zoom-in calls (pink). For each method, darker shades indicate larger input resize ratios (higher resolution, more visual tokens). Notably, InSight-doc shows a clear tendency to push the Pareto frontier toward the upper-left, achieving a more favorable trade-off.
Table 4: Not-answerable F1 scores under low and medium initial-input resolutions (r=0.25 and 0.5).
DUDE
MMLong.
0.25
0.5
0.25
0.5
Qwen3-VL-8B
44.5
57.4
48.5
55.9
Qwen3-VL-8B (w/ zoom)
50.7
55.6
58.7
62.7
InSight-doc-8B (SFT+RL)
69.1
72.4
74.4
75.1
Figure 5: Statistics of our training data.
Table 5: Comparison with visual-retrieval and coarse-to-fine methods. R-f: retriever-free, C2F: coarse-to-fine, Itr: iterative multi-turn evidence acquisition, Rgn: region-level evidence, MMLD.: MMLongBench-Doc, and LDoc.: LongDocURL. Scores are official cross-paper numbers and are not fully controlled for backbone, training data, input resolution, page budget, or evaluation protocol. †With GPT-4o as the main agent; other methods are based on open models of similar sizes.
Method
R-f
C2F
Itr
Rgn
MMLD.
LDoc.
ColPali†
✗
✗
✗
✗
30.8
–
Doc-React†
✗
✗
✓
✗
38.3
–
VDocRAG
✗
✗
✗
✗
18.4
39.8
VRAG-RL
✗
✓
✓
✓
26.6
44.9
CogDoc
✓
✓
✗
✗
33.0
–
DocSeeker
✓
✗
✗
✗
40.1
51.7
Doc-V⋆
✗
✓
✓
✗
42.1
56.3
InSight-doc
✓
✓
✓
✓
57.8
65.6
Figure 6: An example of our SFT data.
Table 6: Trajectory-quality comparison on document VQA benchmarks. Crops denotes the mean number of zoom-in calls. Box denotes the LongDocURL evidence-box hit rate, where a crop must cover at least 50% of an evidence box. A trajectory is redundant if it contains a pair of crops from the same source with IoU ≥0.8. A trajectory is stuck if it exhausts the tool-call budget and ends with at least two consecutive crops on the same page. Area is the union of all cropped regions normalized by the full-page area. Box coverage, redundancy, stuck rate, and area are reported as percentages.
r
Overall
Unanswerable
Crops
Box
Rdn.
Stuck
Area
Crops
Stuck
Qwen3-VL-8B (w/ zoom)
0.25
2.94
27.5
14.1
9.7
15.2
3.03
9.6
0.50
1.66
41.8
6.0
4.4
11.1
1.63
4.6
InSight-doc-8B (SFT)
0.25
2.06
68.1
11.7
5.1
16.4
3.41
10.2
0.50
1.23
70.2
4.1
1.6
12.4
2.08
3.5
InSight-doc-8B (SFT+RL)
0.25
2.34
82.3
5.8
0.1
28.5
2.75
0.0
0.50
1.69
77.0
2.4
0.0
22.5
1.91
0.0
Figure 7: Accuracy vs. efficiency on the longest-document examples from MMLongBench-Doc and LongDocURL (200 examples each). InSight-doc-8B achieves higher accuracy at substantially lower time and token cost than both baselines.
Table 7: Filtering stages for answerable rows before SFT trajectory generation. Each cell reports row count followed by retention relative to the corresponding source-family/DPI source pool. The DPI columns correspond to resize ratios r=0.25,0.35,0.5 from 200-DPI page renders. The arXiv rows bypass prior-only filtering and are therefore carried forward unchanged in the second group; all sources are then included in zoom-free filtering. Unanswerable rows are excluded from this table and described separately.
Filtering stage
Total
50 DPI (r=0.25)
70 DPI (r=0.35)
100 DPI (r=0.5)
All sources
Source pool before filtering
50,903 (100.0%)
25,344 (100.0%)
14,839 (100.0%)
10,720 (100.0%)
After prior-only filtering
44,889 (88.2%)
22,382 (88.3%)
13,153 (88.6%)
9,354 (87.3%)
After zoom-free filtering
26,943 (52.9%)
15,118 (59.7%)
7,370 (49.7%)
4,455 (41.6%)
arXiv
Source pool before filtering
16,949 (100.0%)
8,367 (100.0%)
5,582 (100.0%)
3,000 (100.0%)
After prior-only filtering
16,949 (100.0%)
8,367 (100.0%)
5,582 (100.0%)
3,000 (100.0%)
After zoom-free filtering
11,885 (70.1%)
6,541 (78.2%)
3,654 (65.5%)
1,690 (56.3%)
non-arXiv
Source pool before filtering
33,954 (100.0%)
16,977 (100.0%)
9,257 (100.0%)
7,720 (100.0%)
After prior-only filtering
27,940 (82.3%)
14,015 (82.6%)
7,571 (81.8%)
6,354 (82.3%)
After zoom-free filtering
15,058 (44.3%)
8,577 (50.5%)
3,716 (40.1%)
2,765 (35.8%)
Figure 8: Example of InSight-doc on an unanswerable question, i.e., one whose answer cannot be supported by any evidence in the document.
Table 8: Data-flow summary for filtering, add-ons, and the final SFT/RL split. Each numeric cell is an exact row count; add-on rows with a leading “+” are incremental counts, while other rows are cumulative totals. Within each source-family group, “All” is the sum of answerable (Ans.) and unanswerable (Unans.) rows.
All sources
arXiv
non-arXiv
Stage
All
Ans.
Unans.
All
Ans.
Unans.
All
Ans.
Unans.
Filtering before trajectory generation
Source pool before filtering
62,318
50,903
11,415
22,552
16,949
5,603
39,766
33,954
5,812
After prior-only filtering
56,304
44,889
11,415
22,552
16,949
5,603
33,752
27,940
5,812
After zoom-free filtering
33,502
26,943
6,559
13,436
11,885
1,551
20,066
15,058
5,008
SFT construction
Correct InSight-o3 trajectories
14,717
14,216
501
6,350
6,306
44
8,367
7,910
457
Synthetic unanswerable add-on
+3,196
+0
+3,196
+1,765
+0
+1,765
+1,431
+0
+1,431
Final SFT rows
17,913
14,216
3,697
8,115
6,306
1,809
9,798
7,910
1,888
RL construction
RL candidate pool
18,785
12,727
6,058
7,086
5,579
1,507
11,699
7,148
4,551
After source selection, cleanup, and 24k cap
11,719
7,403
4,316
6,125
4,618
1,507
5,594
2,785
2,809
Synthetic unanswerable add-on
+4,341
+0
+4,341
+2,415
+0
+2,415
+1,926
+0
+1,926
Multiple-choice add-on
+2,176
+2,176
+0
+581
+581
+0
+1,595
+1,595
+0
Structured-document add-on
+1,000
+1,000
+0
+1,000
+1,000
+0
+0
+0
+0
Final RL rows (w/o reweighting)
19,236
10,579
8,657
10,121
6,199
3,922
9,115
4,380
4,735
Figure 9: Examples illustrating InSight-doc’s sequential zoom-in behavior, where it progressively refines target regions until sufficient evidence is collected. The bottom example, taken from MME-RealWorld-Lite, shows that InSight-doc also generalizes to natural-image visual search despite being trained solely on document data.
Table 9: RL sampling targets used by the weighted refill sampler. The weights sum to 1.0.
Sampling target
Weight mass
Answerable rows
86.0%
Unanswerable rows
14.0%
arXiv visually grounded QA
16.04%
arXiv multi-evidence QA
15.39%
arXiv structural rewrites
5.00%
DocVQA
9.97%
DUDE
22.22%
Infographic QA
3.99%
Map metro
15.06%
Map travel
4.75%
Poster QA
7.59%
Figure 10: An example demonstrating that InSight-doc can think and see in a manner similar to humans:adjusting the focus when the initially identified region is imprecise.
Table 10: Resize-ratio distribution of the SFT+RL data.
Dataset
r=0.25
r=0.35
r=0.5
SFT
10,051 (56.1%)
4,913 (27.4%)
2,949 (16.5%)
RL
13,281 (69.0%)
4,394 (22.8%)
1,561 (8.1%)
Figure 11: An example demonstrating that InSight-doc can think and see in a manner similar to humans: actively exploring potential regions that may contain the answer.
Table 11: SFT trajectory-quality metrics. Evidence metrics exclude unanswerable rows and rows without evidence metadata. Page and region hit rates also exclude zero-crop trajectories from their denominators.
Metric
Value
Scope
Rows with page evidence
13,335
Rows with box evidence
6,925
Coverage
Evidence-page hit rate
95.33%
Evidence-region hit rate, coverage ≥0.5
85.00%
Mean max evidence coverage
85.24%
Localization precision
Mean max crop/evidence IoU
52.37%
Crop region-hit rate
65.34%
Crops per evidence-region-hit crop
1.53
Crop area fraction
14.50%
Efficiency and redundancy
Same-source overlap rate, IoU ≥0.8
3.16%
Stuck rate
0.99%
Stop exactly at first region hit
71.14%
Figure 12: Examples demonstrating that InSight-doc can gather evidence from different pages.
Table 12: Judge calibration on a 150-example manually labeled stress set using GPT-5-nano. Metrics are percentages except FP/FN counts.
Judge
Acc.
Prec.
Recall
F1
FP
FN
legacy
84.7
98.1
70.3
81.9
1
22
single-call
87.3
86.7
87.8
87.2
10
9
legacy-v2
94.7
97.1
91.9
94.4
2
6
Table 13: Key hyperparameters for InSight-doc-8B SFT.
Hyperparameter
Value
Initialization
Qwen3-VL-8B-Instruct
# of training examples
17,913
Training steps
1118, 2 epochs
Fine-tuning type
full-parameter fine-tuning
Global batch size
32
Max sequence length
65,536 tokens
Sequence parallel size
4
Learning rate
5×10−6
LR schedule
cosine decay
Warmup ratio
0.05
Minimum learning rate
5×10−7
Optimizer
AdamW
Weight decay
0.01
Gradient clipping
1.0
Vision encoder
frozen
Table 14: Key hyperparameters for InSight-doc-8B RL.
Hyperparameter
Value
Initialization
InSight-doc-8B (SFT)
# of training examples
19,236
RL algorithm
GRPO
Training steps
800
Global batch size
24 prompts
Rollouts per prompt
8
Effective rollout batch size
192 responses
Learning rate
1×10−6
Optimizer
AdamW
KL regularization
low-var. KL, coeff. 0.01
Max prompt length
24,576 tokens
Max response length
8,192 tokens
vLLM max model length
32,768 tokens
Vision encoder
frozen
Sequence parallel size
4
Rollout temperature
0.7
Rollout top-p
0.8
Rollout top-k
20
Rollout presence penalty
1.5
Tool-use limit
10 times
Full-page image max area
35002 pixels
Crop max area
12802 pixels
Reward model
GPT-5-nano judge
Reward weights
accuracy 1.0
Data sampler
weighted random refill
Table 15: Key inference configuration for evaluation.
Configuration
Value
Inference backend
vLLM
vLLM replicas
4
GPUs per replica
1
Number of agent workers
8 (high) or 4 (low)
Worker concurrency
4 (high) or 1 (low)
vLLM max model length
262,144 tokens
Max generated tokens
16,384 tokens
vLLM max batched tokens
32,768 tokens
vLLM max sequences
64
GPU memory utilization
0.8
Prefix caching
enabled
Chunked prefill
enabled
Sampling temperature
0.7
Sampling top-p
0.8
Sampling top-k
20
Presence penalty
1.5
Repetition penalty
1.0
Full-page image max area
35002 pixels
Crop image max area
12802 pixels
Region zoom factor
2.0
Tool parser
Hermes-style tool parser
Max parallel tool calls
1
Tool-use limit
10 times
Context overflow handling
halve image area up to 4×
Table 16: Comparison with Qwen3-VL-8B under no page limit. The models are compared under four initial resolution settings: low (r=0.25, DPI 50), medium-low (r=0.35, DPI 70), and medium (r=0.5, DPI 100), and high (r=0.7, DPI 140). †Page-limited results are included for reference.
MMLongBench-Doc
LongDocURL
Average
Model
0.25
0.35
0.5
0.7
0.25
0.35
0.5
0.7
0.25
0.35
0.5
0.7
Qwen3-VL-8B†
33.7
45.3
51.4
52.0
50.5
63.7
68.4
70.5
42.1
54.5
59.9
61.2
Qwen3-VL-8B
33.0
43.0
49.8
51.1
47.0
57.2
62.6
63.9
40.0
50.1
56.2
57.5
Qwen3-VL-8B (w/ zoom)†
33.2
40.4
48.8
53.1
47.1
58.2
63.8
68.8
40.2
49.3
56.3
60.9
Qwen3-VL-8B (w/ zoom)
33.6
40.5
47.8
49.9
36.7
46.6
57.7
62.2
35.2
43.5
52.7
56.0
InSight-doc-8B (SFT)†
36.5
43.0
48.0
47.3
57.0
61.1
63.7
66.0
46.7
52.0
55.9
56.6
InSight-doc-8B (SFT)
37.9
43.7
45.9
46.7
47.2
54.4
57.2
59.1
42.5
49.1
51.6
52.9
InSight-doc-8B (SFT+RL)†
50.8
55.6
58.6
57.9
63.3
68.4
70.5
71.9
57.0
62.0
64.5
64.9
InSight-doc-8B (SFT+RL)
50.0
55.1
57.8
58.5
57.3
63.2
65.6
67.0
53.7
59.1
61.7
62.7
Table 17: Expanded comparison with retrieval-based, coarse-to-fine, iterative, and structured document reasoning methods. Scores are official cross-paper numbers and are used for positioning rather than controlled head-to-head comparison. R-f (retriever-free) means no external page/document retriever is used. C2F (coarse-to-fine) means the method starts from coarse document view and then acquires finer evidence. Itr (iterative) means explicit multi-turn evidence acquisition. Rgn (region) means region-level evidence acquisition rather than page-level. MPDoc., MMLD., and LDoc. refer to MP-DocVQA, MMLongBench-Doc, and LongDocURL, respectively. †Prior work commonly reports ANLS on DUDE and MP-DocVQA, while our InSight-doc results are evaluated with the same LLM-as-judge protocol used for our main experiments across benchmarks.
Method
Backbone
Param.
R-f
C2F
Itr
Rgn
DUDE
MPDoc.
MMLD.
LDoc.
Retrieval-based methods
GPT-4o + ColPali
GPT-4o
–
✗
✗
✗
✗
–
–
30.8
–
CREAM
Pix2Struct + LLaMA2
7B
✗
✓
✗
✗
52.5
74.3
–
–
M3DocRAG
Qwen2-VL
7B
✗
✗
✗
✗
39.5
84.4
21.0
35.1
VisRAG
MiniCPM-V 2.6
8B
✗
✗
✗
✗
43.1
–
18.8
41.9
SV-RAG
InternVL2
4B
✗
✗
✗
✗
45.0
71.0
23.0
–
VDocRAG
Phi3-Vision
4B
✗
✗
✗
✗
44.0
62.6
18.4
39.8
MoLoRAG
Qwen2.5-VL
7B
✗
✗
✗
✗
–
–
41.0
51.9
URaG
Qwen2.5-VL
7B
✗
✓
✗
✗
57.6
88.2
33.8
52.2
Iterative, coarse-to-fine, and structured reasoning methods
Doc-React
GPT-4o
–
✗
✗
✓
✗
–
–
38.3
–
VRAG-RL
Qwen2.5-VL
7B
✗
✓
✓
✓
–
–
26.6
44.9
CogDoc
Qwen2.5-VL
7B
✓
✓
✗
✗
46.2
75.0
33.0
–
DocR1
Qwen2.5-VL
7B
✓
✓
✗
✗
54.4
87.5
–
–
DocSeeker
Qwen2.5-VL
7B
✓
✗
✗
✗
57.4
86.2
40.1
51.7
Doc-V⋆
Qwen2.5-VL
7B
✗
✓
✓
✗
64.5
86.2
42.1
56.3
MM-Doc-R1
Qwen3 + Qwen2.5-VL
8B
✗
✓
✓
✗
–
–
49.7
–
InSight-doc (r=0.25)
Qwen3-VL
8B
✓
✓
✓
✓
†70.1†
†83.4†
50.0
57.3
InSight-doc (r=0.35)
Qwen3-VL
8B
✓
✓
✓
✓
†72.1†
†86.3†
55.1
63.2
InSight-doc (r=0.5)
Qwen3-VL
8B
✓
✓
✓
✓
†73.8†
†87.6†
57.8
65.6
InSight-doc (r=0.7)
Qwen3-VL
8B
✓
✓
✓
✓
†73.8†
†88.2†
58.5
67.0
Table 18: Controlled proxy comparison with retrieval-assisted page-level evidence selection on the 200 longest documents from each of MMLongBench-Doc and LongDocURL. The ColQwen2.5 baseline retrieves the top-K pages and feeds them to Qwen3-VL-8B-Instruct, approximating the external-retriever setting used by Doc-V⋆ while controlling the answer-generation backbone.
Method
Setting
Acc. (%)
Avg. Tokens
Qwen3-VL-8B + ColQwen2.5
Top-8 pages
43.4
∼28K
InSight-doc-8B
50 DPI
48.8
∼22K
Qwen3-VL-8B + ColQwen2.5
Top-16 pages
49.1
∼57K
InSight-doc-8B
70 DPI
56.2
∼41K
Qwen3-VL-8B + ColQwen2.5
Top-32 pages
53.5
∼104K
InSight-doc-8B
100 DPI
57.9
∼81K
실제로 확인된 결과
문서 질의응답 4개 벤치마크에서 저해상도 입력 기준 평균 정확도가 66.9%로 기반 모델보다 16.4점, 중간 해상도에서는 72.6%로 4.3점 높았다.
긴 문서 최장 사례에서 InSight-doc(70DPI)은 42.4천 토큰으로 56.2% 정확도를 냈고, 기반 모델(140DPI)은 136.8천 토큰으로 53.2% 정확도에 그쳐, 토큰은 69% 줄이면서 정확도도 더 높았다.
MMLongBench-Doc에서 InSight-doc은 9.3초에 55.6% 정확도, 기반 모델(140DPI)은 21.2초에 52.0% 정확도를 기록해 지연시간이 크게 줄었다.
답할 수 없는 질문에 대한 F1 점수가 저해상도에서 DUDE 24.6점, MMLongBench-Doc 25.9점 향상됐다.
강화학습(RL)을 추가한 결과 지도학습(SFT)만 쓴 경우보다 평균 정확도가 저해상도에서 56.6%→66.9%, 중간 해상도에서 64.6%→72.6%로 더 올랐다.
어디에 쓸 수 있나
연구 논문, 재무 보고서 등 페이지 수가 많은 문서를 다루는 질의응답 시스템에서 비용과 지연 시간을 줄이면서 정확도를 유지하려는 경우
답할 수 없는 질문에 대해 근거 없이 답을 지어내는 것(할루시네이션)을 줄여야 하는 문서 기반 챗봇이나 검토 도구
외부 검색기(retriever) 없이 모델 스스로 필요한 영역을 찾아 확인하는 구조를 참고해 비슷한 시각 검색 에이전트를 설계하려는 경우
한계와 남은 검증
실험은 Qwen3-VL-8B-Instruct라는 하나의 기반 모델에만 적용됐고, 다른 제공사의 모델이나 더 최신 모델에 대한 검증은 아직 없다.
강화학습에는 단순한 정답 여부(이진 정확도)만 보상으로 썼고, 더 발전된 강화학습 방법이나 보상 설계는 시도되지 않았다.
Doc-V★ 같은 가장 근접한 경쟁 방법은 학습 데이터·체크포인트·코드가 공개되지 않아 완전히 통제된 비교를 하지 못했고, 인용된 비교표의 수치는 백본·해상도·평가 방식이 서로 달라 참고용으로만 해석해야 한다.
페이지 수 제한(최대 40쪽)을 둔 평가에서는 실제 긴 문서의 난이도가 일부 가려질 수 있으며, 제한을 없앤 평가에서는 특히 LongDocURL에서 정확도가 더 떨어지는 경향이 확인됐다.
왜 중요한가
긴 보고서나 논문처럼 페이지 수가 많은 문서를 AI가 다룰 때 드는 비용과 시간, 그리고 근거 없이 답을 지어내는 문제를 동시에 줄일 수 있는 실용적인 접근을 보여준다. 문서를 다루는 실무 도구를 만들 때 참고할 수 있는 구체적인 학습 데이터 구성법과 성능 수치를 함께 공개했다.
이 논문의 용어
컨텍스트 로트(context rot) · 입력이 길어질수록 AI 모델의 성능이 급격히 나빠지는 현상
줌인(zoom-in) · 이미지의 특정 영역을 원본 고해상도에서 잘라내 확대해서 보는 동작
SFT (지도 미세조정) · 정답이 포함된 시범 데이터를 모델이 그대로 따라 배우게 하는 학습 방식
RL (강화학습) · 정답 여부 같은 보상 신호만으로 모델이 스스로 행동을 개선하게 하는 학습 방식
