InSight-doc: Agentic Visual Perception for Long-Document Understanding

arXiv:2608.106282026-08-10

AI先低分辨率浏览长文档,再自己放大需要细看的区域,回答更准更快

InSight-doc是一个处理长文档的AI智能体,它先用低分辨率整体浏览文档,再自主判断哪些区域需要放大,直接从原始高分辨率页面裁剪出这些区域作为证据,整个过程不依赖任何外部检索工具。研究者用示范轨迹数据先做模仿学习,再用只看答案对错的强化学习进一步训练一个80亿参数模型,结果文档问答准确率比基础模型提升4.3到16.4个百分点。在长文档上,它还把编造答案(幻觉)的比例降低超过40%,推理延迟降低41%到68%。

METAL MEDIA 解读图

InSight-doc的低分辨率到放大推理流程

证据状态已报告实测结果

低分辨率全文档输入文档所有页面先以较低分辨率(如50DPI)提供给模型,大幅减少视觉token数量
思考并调用放大工具模型用<think>进行推理,并通过<tool_call>指定要放大的页面、区域和边界框
裁剪高分辨率区域从原始高分辨率页面裁剪出请求的区域,作为新的视觉证据加入模型的上下文
重复收集证据模型可以多次放大,必要时跨越不同页面,直到收集到足够证据
生成最终答案证据充分后模型输出<answer>,若找不到充分依据则标明问题无法回答

这是 METAL MEDIA 制作的解读图，并非论文作者提供的原图。

他们做了什么

把整份文档以高分辨率全部输入AI模型速度慢,而低分辨率输入又会丢失细节,这是长文档理解的固有矛盾。InSight-doc的做法是先以低分辨率输入,再根据需要从原始高分辨率页面裁剪放大特定区域,借此缓解这一矛盾。
为了训练这种行为,研究者先过滤掉不看文档也能答对、以及低分辨率下已经能答对的问题,再用两个AI智能体(负责推理的reasoner和负责定位的searcher)协作生成约1.79万条演示放大位置的轨迹数据,外加1.92万条更难的强化学习样本。模型先模仿这些演示轨迹完成监督微调(SFT),再用仅根据答案对错给出奖励的强化学习(RL)继续训练。
在DUDE、MP-DocVQA等标准文档问答和MMLongBench-Doc、LongDocURL等长文档问答(平均49到86页)上,低分辨率输入下InSight-doc-8B的平均准确率比基础模型Qwen3-VL-8B高16.4个百分点,中等分辨率下高4.3个百分点。
针对文档中根本没有依据的'无法回答'问题,模型不再随意编造答案的能力显著提升,F1分数比基础模型最多提高25.9个百分点。
在保持甚至提升准确率的同时,最长文档样本上的token用量最多减少约69%,推理延迟最多降低约71%。

Figure 1: InSight-doc substantially reduces hallucination, sequence length, and latency while improving accuracy on long-document VQA. Hallucination rate is measured on unanswerable questions as the rate of non-abstaining answers. More details in Appendix F.1.

Table 1: Relative-latency upper bound for representative parameters. Here, n is the number of zoom-in tool calls.
r	n=0	n=1	n=2	n=3	n=4
0.25	0.046	0.124	0.238	0.389	0.576
0.35	0.090	0.189	0.325	0.498	0.707
0.50	0.190	0.336	0.519	0.738	0.994

Figure 2: Illustration of InSight-doc on a long-document VQA example. Starting from a low-resolution overview, InSight-doc performs an interleaved multimodal chain-of-thought: each round emits a thought (<think>) and a zoom-in tool call (<tool_call>) specifying img_idx, label, and bbox; the cropped high-resolution region is then appended as visual evidence. After multiple rounds of active perception, the model produces the final <answer>. Images shown above are not in the exact scales seen by the model. More examples can be found in Appendix G.

Table 2: Comparison with frontier models on medium-to-long document VQA. The models are compared under two initial resolution settings: low (r=0.25, DPI 50) and medium (r=0.5, DPI 100). The inference mode “E2E” means single-turn QA without tool use, while “Agent” allows multi-turn zoom-in tool use. LLM-as-judge accuracy is reported on all benchmarks. Input page limit is set to 40 for all models due to API limit on some closed models. The limit only affects MMLongBench-Doc and LongDocURL. The unlimited results can be found in Appendix F.2.
		DUDE	MP-DVQA	MMLong.	LongDoc.	Average
		0.25	0.5	0.25	0.5	0.25	0.5	0.25	0.5	0.25	0.5
Closed proprietary models
GPT-5.4-nano	E2E	52.8	66.0	64.2	83.4	33.6	52.2	54.4	72.7	51.2	68.6
GPT-5.4-mini	E2E	63.2	71.6	78.3	88.9	46.4	58.8	69.3	77.6	64.3	74.2
GPT-5-mini	E2E	63.9	70.0	81.3	89.5	48.0	57.2	70.3	80.4	65.9	74.3
Gemini-3.1-Flash-lite	E2E	70.2	70.8	87.5	89.4	57.4	58.3	75.8	75.7	72.7	73.5
Gemini-3-Flash	E2E	72.0	72.4	89.3	89.9	62.4	61.8	77.6	76.3	75.3	75.1
Open models
InternVL3-8B	E2E	52.7	62.2	66.5	84.7	13.6	33.5	26.7	42.9	39.9	55.8
GLM-4.6V-Flash (9B)	E2E	40.7	53.4	50.9	72.5	14.8	15.0	24.6	26.0	32.8	41.7
Qwen3-VL-8B	E2E	52.9	68.5	65.1	84.9	33.7	51.4	50.5	68.4	50.5	68.3
Qwen3-VL-8B (w/ zoom)	Agent	55.1	66.4	66.1	82.6	33.2	48.8	47.1	63.8	50.4	65.4
InSight-doc models
InSight-doc-8B (SFT)	Agent	60.8	67.2	72.2	79.5	36.5	48.0	57.0	63.7	56.6	64.6
InSight-doc-8B (SFT+RL)	Agent	70.1	73.8	83.4	87.6	50.8	58.6	63.3	70.5	66.9	72.6
Δ w.r.t. Qwen3-VL-8B		+17.2	+05.3	+18.3	+02.7	+17.1	+07.2	+12.8	+02.1	+16.4	+04.3

Figure 3: Overview of data construction pipeline. The top panel shows a three-stage filtering and CoT construction process: Stage 1 discards questions answerable without the document, Stage 2 discards questions already answerable at low DPI without zoom, and Stage 3 uses InSight-o3 to construct zoom-in CoTs for the remaining items, routing successes to SFT and failures to RL. The bottom panel illustrates the InSight-o3 two-agent trajectory, where the vReasoner and vSearcher iteratively produce reasoning steps, zoom-in requests, bounding boxes, and crops, which are then merged into a single flat multimodal CoT used as the imitation target for InSight-doc.

Table 3: Comparison with frontier models on general high-resolution VQA. LLM-as-judge accuracy is reported under two initial resolution settings: r=0.25 and r=0.5. The inference modes follow Table 2.
	MME-RWℓ	O3-Bench
	0.25	0.5	0.25	0.5
GPT-5.4-nano	40.3	47.9	29.6	32.8
GPT-5.4-mini	47.4	57.2	40.9	56.2
Gemini-3.1-Flash-lite	47.8	51.8	44.6	51.0
Qwen3-VL-8B	41.0	49.2	22.3	35.1
Qwen3-VL-8B (w/ zoom)	42.5	50.6	20.3	35.4
InSight-doc-8B (SFT+RL)	48.2	52.9	24.1	43.8

Figure 4: Accuracy vs. efficiency on four multi-page document VQA benchmarks. We compare InSight-doc (ours, green) with Qwen3-VL-8B without zoom-in calls (purple) and with zoom-in calls (pink). For each method, darker shades indicate larger input resize ratios (higher resolution, more visual tokens). Notably, InSight-doc shows a clear tendency to push the Pareto frontier toward the upper-left, achieving a more favorable trade-off.

Table 4: Not-answerable F1 scores under low and medium initial-input resolutions (r=0.25 and 0.5).
	DUDE	MMLong.
	0.25	0.5	0.25	0.5
Qwen3-VL-8B	44.5	57.4	48.5	55.9
Qwen3-VL-8B (w/ zoom)	50.7	55.6	58.7	62.7
InSight-doc-8B (SFT+RL)	69.1	72.4	74.4	75.1

Figure 5: Statistics of our training data.

Table 5: Comparison with visual-retrieval and coarse-to-fine methods. R-f: retriever-free, C2F: coarse-to-fine, Itr: iterative multi-turn evidence acquisition, Rgn: region-level evidence, MMLD.: MMLongBench-Doc, and LDoc.: LongDocURL. Scores are official cross-paper numbers and are not fully controlled for backbone, training data, input resolution, page budget, or evaluation protocol. †With GPT-4o as the main agent; other methods are based on open models of similar sizes.
Method	R-f	C2F	Itr	Rgn	MMLD.	LDoc.
ColPali†	✗	✗	✗	✗	30.8	–
Doc-React†	✗	✗	✓	✗	38.3	–
VDocRAG	✗	✗	✗	✗	18.4	39.8
VRAG-RL	✗	✓	✓	✓	26.6	44.9
CogDoc	✓	✓	✗	✗	33.0	–
DocSeeker	✓	✗	✗	✗	40.1	51.7
Doc-V⋆	✗	✓	✓	✗	42.1	56.3
InSight-doc	✓	✓	✓	✓	57.8	65.6

Table 6: Trajectory-quality comparison on document VQA benchmarks. Crops denotes the mean number of zoom-in calls. Box denotes the LongDocURL evidence-box hit rate, where a crop must cover at least 50% of an evidence box. A trajectory is redundant if it contains a pair of crops from the same source with IoU ≥0.8. A trajectory is stuck if it exhausts the tool-call budget and ends with at least two consecutive crops on the same page. Area is the union of all cropped regions normalized by the full-page area. Box coverage, redundancy, stuck rate, and area are reported as percentages.
r	Overall	Unanswerable
Crops	Box	Rdn.	Stuck	Area	Crops	Stuck
Qwen3-VL-8B (w/ zoom)
0.25	2.94	27.5	14.1	9.7	15.2	3.03	9.6
0.50	1.66	41.8	6.0	4.4	11.1	1.63	4.6
InSight-doc-8B (SFT)
0.25	2.06	68.1	11.7	5.1	16.4	3.41	10.2
0.50	1.23	70.2	4.1	1.6	12.4	2.08	3.5
InSight-doc-8B (SFT+RL)
0.25	2.34	82.3	5.8	0.1	28.5	2.75	0.0
0.50	1.69	77.0	2.4	0.0	22.5	1.91	0.0

Figure 7: Accuracy vs. efficiency on the longest-document examples from MMLongBench-Doc and LongDocURL (200 examples each). InSight-doc-8B achieves higher accuracy at substantially lower time and token cost than both baselines.

Table 7: Filtering stages for answerable rows before SFT trajectory generation. Each cell reports row count followed by retention relative to the corresponding source-family/DPI source pool. The DPI columns correspond to resize ratios r=0.25,0.35,0.5 from 200-DPI page renders. The arXiv rows bypass prior-only filtering and are therefore carried forward unchanged in the second group; all sources are then included in zoom-free filtering. Unanswerable rows are excluded from this table and described separately.
Filtering stage	Total	50 DPI (r=0.25)	70 DPI (r=0.35)	100 DPI (r=0.5)
All sources
Source pool before filtering	50,903 (100.0%)	25,344 (100.0%)	14,839 (100.0%)	10,720 (100.0%)
After prior-only filtering	44,889 (88.2%)	22,382 (88.3%)	13,153 (88.6%)	9,354 (87.3%)
After zoom-free filtering	26,943 (52.9%)	15,118 (59.7%)	7,370 (49.7%)	4,455 (41.6%)
arXiv
Source pool before filtering	16,949 (100.0%)	8,367 (100.0%)	5,582 (100.0%)	3,000 (100.0%)
After prior-only filtering	16,949 (100.0%)	8,367 (100.0%)	5,582 (100.0%)	3,000 (100.0%)
After zoom-free filtering	11,885 (70.1%)	6,541 (78.2%)	3,654 (65.5%)	1,690 (56.3%)
non-arXiv
Source pool before filtering	33,954 (100.0%)	16,977 (100.0%)	9,257 (100.0%)	7,720 (100.0%)
After prior-only filtering	27,940 (82.3%)	14,015 (82.6%)	7,571 (81.8%)	6,354 (82.3%)
After zoom-free filtering	15,058 (44.3%)	8,577 (50.5%)	3,716 (40.1%)	2,765 (35.8%)

Figure 8: Example of InSight-doc on an unanswerable question, i.e., one whose answer cannot be supported by any evidence in the document.

Table 8: Data-flow summary for filtering, add-ons, and the final SFT/RL split. Each numeric cell is an exact row count; add-on rows with a leading “+” are incremental counts, while other rows are cumulative totals. Within each source-family group, “All” is the sum of answerable (Ans.) and unanswerable (Unans.) rows.
	All sources	arXiv	non-arXiv
Stage	All	Ans.	Unans.	All	Ans.	Unans.	All	Ans.	Unans.
Filtering before trajectory generation
Source pool before filtering	62,318	50,903	11,415	22,552	16,949	5,603	39,766	33,954	5,812
After prior-only filtering	56,304	44,889	11,415	22,552	16,949	5,603	33,752	27,940	5,812
After zoom-free filtering	33,502	26,943	6,559	13,436	11,885	1,551	20,066	15,058	5,008
SFT construction
Correct InSight-o3 trajectories	14,717	14,216	501	6,350	6,306	44	8,367	7,910	457
Synthetic unanswerable add-on	+3,196	+0	+3,196	+1,765	+0	+1,765	+1,431	+0	+1,431
Final SFT rows	17,913	14,216	3,697	8,115	6,306	1,809	9,798	7,910	1,888
RL construction
RL candidate pool	18,785	12,727	6,058	7,086	5,579	1,507	11,699	7,148	4,551
After source selection, cleanup, and 24k cap	11,719	7,403	4,316	6,125	4,618	1,507	5,594	2,785	2,809
Synthetic unanswerable add-on	+4,341	+0	+4,341	+2,415	+0	+2,415	+1,926	+0	+1,926
Multiple-choice add-on	+2,176	+2,176	+0	+581	+581	+0	+1,595	+1,595	+0
Structured-document add-on	+1,000	+1,000	+0	+1,000	+1,000	+0	+0	+0	+0
Final RL rows (w/o reweighting)	19,236	10,579	8,657	10,121	6,199	3,922	9,115	4,380	4,735

Figure 9: Examples illustrating InSight-doc’s sequential zoom-in behavior, where it progressively refines target regions until sufficient evidence is collected. The bottom example, taken from MME-RealWorld-Lite, shows that InSight-doc also generalizes to natural-image visual search despite being trained solely on document data.

Table 9: RL sampling targets used by the weighted refill sampler. The weights sum to 1.0.
Sampling target	Weight mass
Answerable rows	86.0%
Unanswerable rows	14.0%
arXiv visually grounded QA	16.04%
arXiv multi-evidence QA	15.39%
arXiv structural rewrites	5.00%
DocVQA	9.97%
DUDE	22.22%
Infographic QA	3.99%
Map metro	15.06%
Map travel	4.75%
Poster QA	7.59%

Figure 10: An example demonstrating that InSight-doc can think and see in a manner similar to humans:adjusting the focus when the initially identified region is imprecise.

Table 10: Resize-ratio distribution of the SFT+RL data.
Dataset	r=0.25	r=0.35	r=0.5
SFT	10,051 (56.1%)	4,913 (27.4%)	2,949 (16.5%)
RL	13,281 (69.0%)	4,394 (22.8%)	1,561 (8.1%)

Figure 11: An example demonstrating that InSight-doc can think and see in a manner similar to humans: actively exploring potential regions that may contain the answer.

Table 11: SFT trajectory-quality metrics. Evidence metrics exclude unanswerable rows and rows without evidence metadata. Page and region hit rates also exclude zero-crop trajectories from their denominators.
Metric	Value
Scope
Rows with page evidence	13,335
Rows with box evidence	6,925
Coverage
Evidence-page hit rate	95.33%
Evidence-region hit rate, coverage ≥0.5	85.00%
Mean max evidence coverage	85.24%
Localization precision
Mean max crop/evidence IoU	52.37%
Crop region-hit rate	65.34%
Crops per evidence-region-hit crop	1.53
Crop area fraction	14.50%
Efficiency and redundancy
Same-source overlap rate, IoU ≥0.8	3.16%
Stuck rate	0.99%
Stop exactly at first region hit	71.14%

Figure 12: Examples demonstrating that InSight-doc can gather evidence from different pages.

Table 12: Judge calibration on a 150-example manually labeled stress set using GPT-5-nano. Metrics are percentages except FP/FN counts.
Judge	Acc.	Prec.	Recall	F1	FP	FN
legacy	84.7	98.1	70.3	81.9	1	22
single-call	87.3	86.7	87.8	87.2	10	9
legacy-v2	94.7	97.1	91.9	94.4	2	6

Table 13: Key hyperparameters for InSight-doc-8B SFT.
Hyperparameter	Value
Initialization	Qwen3-VL-8B-Instruct
# of training examples	17,913
Training steps	1118, 2 epochs
Fine-tuning type	full-parameter fine-tuning
Global batch size	32
Max sequence length	65,536 tokens
Sequence parallel size	4
Learning rate	5×10−6
LR schedule	cosine decay
Warmup ratio	0.05
Minimum learning rate	5×10−7
Optimizer	AdamW
Weight decay	0.01
Gradient clipping	1.0
Vision encoder	frozen

Table 14: Key hyperparameters for InSight-doc-8B RL.
Hyperparameter	Value
Initialization	InSight-doc-8B (SFT)
# of training examples	19,236
RL algorithm	GRPO
Training steps	800
Global batch size	24 prompts
Rollouts per prompt	8
Effective rollout batch size	192 responses
Learning rate	1×10−6
Optimizer	AdamW
KL regularization	low-var. KL, coeff. 0.01
Max prompt length	24,576 tokens
Max response length	8,192 tokens
vLLM max model length	32,768 tokens
Vision encoder	frozen
Sequence parallel size	4
Rollout temperature	0.7
Rollout top-p	0.8
Rollout top-k	20
Rollout presence penalty	1.5
Tool-use limit	10 times
Full-page image max area	35002 pixels
Crop max area	12802 pixels
Reward model	GPT-5-nano judge
Reward weights	accuracy 1.0
Data sampler	weighted random refill

Table 15: Key inference configuration for evaluation.
Configuration	Value
Inference backend	vLLM
vLLM replicas	4
GPUs per replica	1
Number of agent workers	8 (high) or 4 (low)
Worker concurrency	4 (high) or 1 (low)
vLLM max model length	262,144 tokens
Max generated tokens	16,384 tokens
vLLM max batched tokens	32,768 tokens
vLLM max sequences	64
GPU memory utilization	0.8
Prefix caching	enabled
Chunked prefill	enabled
Sampling temperature	0.7
Sampling top-p	0.8
Sampling top-k	20
Presence penalty	1.5
Repetition penalty	1.0
Full-page image max area	35002 pixels
Crop image max area	12802 pixels
Region zoom factor	2.0
Tool parser	Hermes-style tool parser
Max parallel tool calls	1
Tool-use limit	10 times
Context overflow handling	halve image area up to 4×

Table 16: Comparison with Qwen3-VL-8B under no page limit. The models are compared under four initial resolution settings: low (r=0.25, DPI 50), medium-low (r=0.35, DPI 70), and medium (r=0.5, DPI 100), and high (r=0.7, DPI 140). †Page-limited results are included for reference.
	MMLongBench-Doc	LongDocURL	Average
Model	0.25	0.35	0.5	0.7	0.25	0.35	0.5	0.7	0.25	0.35	0.5	0.7
Qwen3-VL-8B†	33.7	45.3	51.4	52.0	50.5	63.7	68.4	70.5	42.1	54.5	59.9	61.2
Qwen3-VL-8B	33.0	43.0	49.8	51.1	47.0	57.2	62.6	63.9	40.0	50.1	56.2	57.5
Qwen3-VL-8B (w/ zoom)†	33.2	40.4	48.8	53.1	47.1	58.2	63.8	68.8	40.2	49.3	56.3	60.9
Qwen3-VL-8B (w/ zoom)	33.6	40.5	47.8	49.9	36.7	46.6	57.7	62.2	35.2	43.5	52.7	56.0
InSight-doc-8B (SFT)†	36.5	43.0	48.0	47.3	57.0	61.1	63.7	66.0	46.7	52.0	55.9	56.6
InSight-doc-8B (SFT)	37.9	43.7	45.9	46.7	47.2	54.4	57.2	59.1	42.5	49.1	51.6	52.9
InSight-doc-8B (SFT+RL)†	50.8	55.6	58.6	57.9	63.3	68.4	70.5	71.9	57.0	62.0	64.5	64.9
InSight-doc-8B (SFT+RL)	50.0	55.1	57.8	58.5	57.3	63.2	65.6	67.0	53.7	59.1	61.7	62.7

Table 17: Expanded comparison with retrieval-based, coarse-to-fine, iterative, and structured document reasoning methods. Scores are official cross-paper numbers and are used for positioning rather than controlled head-to-head comparison. R-f (retriever-free) means no external page/document retriever is used. C2F (coarse-to-fine) means the method starts from coarse document view and then acquires finer evidence. Itr (iterative) means explicit multi-turn evidence acquisition. Rgn (region) means region-level evidence acquisition rather than page-level. MPDoc., MMLD., and LDoc. refer to MP-DocVQA, MMLongBench-Doc, and LongDocURL, respectively. †Prior work commonly reports ANLS on DUDE and MP-DocVQA, while our InSight-doc results are evaluated with the same LLM-as-judge protocol used for our main experiments across benchmarks.
Method	Backbone	Param.	R-f	C2F	Itr	Rgn	DUDE	MPDoc.	MMLD.	LDoc.
Retrieval-based methods
GPT-4o + ColPali	GPT-4o	–	✗	✗	✗	✗	–	–	30.8	–
CREAM	Pix2Struct + LLaMA2	7B	✗	✓	✗	✗	52.5	74.3	–	–
M3DocRAG	Qwen2-VL	7B	✗	✗	✗	✗	39.5	84.4	21.0	35.1
VisRAG	MiniCPM-V 2.6	8B	✗	✗	✗	✗	43.1	–	18.8	41.9
SV-RAG	InternVL2	4B	✗	✗	✗	✗	45.0	71.0	23.0	–
VDocRAG	Phi3-Vision	4B	✗	✗	✗	✗	44.0	62.6	18.4	39.8
MoLoRAG	Qwen2.5-VL	7B	✗	✗	✗	✗	–	–	41.0	51.9
URaG	Qwen2.5-VL	7B	✗	✓	✗	✗	57.6	88.2	33.8	52.2
Iterative, coarse-to-fine, and structured reasoning methods
Doc-React	GPT-4o	–	✗	✗	✓	✗	–	–	38.3	–
VRAG-RL	Qwen2.5-VL	7B	✗	✓	✓	✓	–	–	26.6	44.9
CogDoc	Qwen2.5-VL	7B	✓	✓	✗	✗	46.2	75.0	33.0	–
DocR1	Qwen2.5-VL	7B	✓	✓	✗	✗	54.4	87.5	–	–
DocSeeker	Qwen2.5-VL	7B	✓	✗	✗	✗	57.4	86.2	40.1	51.7
Doc-V⋆	Qwen2.5-VL	7B	✗	✓	✓	✗	64.5	86.2	42.1	56.3
MM-Doc-R1	Qwen3 + Qwen2.5-VL	8B	✗	✓	✓	✗	–	–	49.7	–
InSight-doc (r=0.25)	Qwen3-VL	8B	✓	✓	✓	✓	†70.1†	†83.4†	50.0	57.3
InSight-doc (r=0.35)	Qwen3-VL	8B	✓	✓	✓	✓	†72.1†	†86.3†	55.1	63.2
InSight-doc (r=0.5)	Qwen3-VL	8B	✓	✓	✓	✓	†73.8†	†87.6†	57.8	65.6
InSight-doc (r=0.7)	Qwen3-VL	8B	✓	✓	✓	✓	†73.8†	†88.2†	58.5	67.0

Table 18: Controlled proxy comparison with retrieval-assisted page-level evidence selection on the 200 longest documents from each of MMLongBench-Doc and LongDocURL. The ColQwen2.5 baseline retrieves the top-K pages and feeds them to Qwen3-VL-8B-Instruct, approximating the external-retriever setting used by Doc-V⋆ while controlling the answer-generation backbone.
Method	Setting	Acc. (%)	Avg. Tokens
Qwen3-VL-8B + ColQwen2.5	Top-8 pages	43.4	∼28K
InSight-doc-8B	50 DPI	48.8	∼22K
Qwen3-VL-8B + ColQwen2.5	Top-16 pages	49.1	∼57K
InSight-doc-8B	70 DPI	56.2	∼41K
Qwen3-VL-8B + ColQwen2.5	Top-32 pages	53.5	∼104K
InSight-doc-8B	100 DPI	57.9	∼81K

研究结果

在四个文档问答基准上,低分辨率输入下平均准确率达66.9%,比基础模型高16.4个百分点;中等分辨率下达72.6%,高4.3个百分点。
在最长文档子集上,InSight-doc在70DPI下用4.24万token达到56.2%准确率,而140DPI的基础模型用13.68万token只有53.2%准确率,token减少69%的同时准确率反而更高。
在MMLongBench-Doc上,InSight-doc用9.3秒达到55.6%准确率,140DPI基础模型用21.2秒只有52.0%准确率。
低分辨率下,无法回答问题的F1分数在DUDE上提升24.6个百分点,在MMLongBench-Doc上提升25.9个百分点。
在SFT基础上加入RL后,平均准确率在低分辨率下从56.6%提升到66.9%,在中等分辨率下从64.6%提升到72.6%。

可应用场景

处理研究论文、财务报告等图文丰富的长文档问答系统,需要在控制成本和延迟的同时保持准确率
需要避免在证据不足时编造答案的文档聊天机器人或审阅工具
参考其不依赖外部检索器、由模型自主定位所需区域的结构,设计类似的视觉搜索智能体

局限与待验证事项

实验仅在Qwen3-VL-8B-Instruct这一个基础模型上进行,尚未验证其他厂商或更新模型的效果。
强化学习仅使用简单的答案对错二元奖励,未尝试更先进的强化学习方法或奖励设计。
最接近的竞争方法Doc-V★未公开训练数据、模型权重和代码,因此与其及其他先前方法的比较使用的是跨论文数字,骨干模型、分辨率、评测方式各不相同,只能作为参考而非严格对照。
部分基准评测设置了页数上限(最多40页),取消上限后准确率下降更明显,尤其是LongDocURL,说明有页数上限的评测可能低估了真正长文档的难度。

为什么重要

这项工作展示了一种让AI处理长篇图文文档(报告、论文等)时既能省钱省时,又能减少凭空编造答案的实用方法。它公开了具体的数据构建方式和实测数字,可供构建文档理解工具的从业者参考。

本文术语

语境退化(context rot) · 输入内容越长,AI模型表现越差的现象
放大(zoom-in) · 从原始高分辨率图像中裁剪并放大特定区域的操作
SFT(监督微调) · 让模型直接模仿带标注的示范样本进行学习的训练方式
RL(强化学习) · 仅凭奖励信号(如答案对错)让模型自主改进行为的训练方式
GRPO · 本研究中使用的强化学习算法名称
LLM-as-judge · 让另一个AI模型来判断答案是否正确的评测方式

论文原文摘要（英文）

Long-document understanding often requires reasoning over many visually rich pages, making inference costly and prone to context rot. In this work, we propose InSight-doc, an agentic visual perception framework that treats visual resolution as an adaptive reasoning-time resource. InSight-doc starts from low resolution and selectively zooms into high-resolution regions for finer evidence, without relying on any external retriever. To train such an agent, we construct an active-perception corpus of 17.9K high-quality SFT examples with region-level zoom-in trajectories, accompanied by 19.2K hard RL examples. Through SFT+RL, InSight-doc-8B improves the baseline by 4.3--16.4 accuracy points over document VQA benchmarks. On long documents, it reduces hallucination by more than 40% and inference latency by 41%--68% while maintaining an accuracy lead. Our code, datasets, and model are released at https://github.com/m-Just/InSight-doc .

作者 · Kaican Li

在 arXiv 阅读

METAL MEDIA 最新报道

图片来源： Kaican Li et al., arXiv:2608.10628, CC BY 4.0