DistilVDR: A Compact End-to-End Visual Document Retriever via Dual-Student Distillation

arXiv:2608.106362026-08-10

A big 8B document-search AI gets copied down into a 524M-parameter compact model

DistilVDR trains a 524M-parameter student to reproduce the embedding space of an 8B vision-language teacher, building a compact system that retrieves document images from text queries. Trained purely with a cosine-similarity loss and no labels or contrastive learning, it reaches 86.9% of the teacher's average retrieval score on the ViDoRe benchmark while storing 15.6 times less index data and indexing over ten times faster than the strongest prior sub-1B model. Two released variants, HiRes and Fast, trade off quality against speed by changing how many image tiles the document side processes.

METAL MEDIA explanatory visual

Two students distilled independently from one frozen teacher

Evidence statusMeasured results reported

Frozen 8B teacherPrecomputes 4096-dim target embeddings for queries and document images once; never runs during student training
Document student (454M)Splits page images into tiles, encodes them with an InternViT visual encoder, re-contextualises with a ModernBERT text backbone, and matches the teacher's document vector via cosine loss
Query student (70M)Encodes text queries with DistilBERT and independently matches the teacher's query vector via cosine loss
Single-vector retrievalRanks documents by dot product between the two students' vectors, avoiding the storage and latency cost of multi-vector matching
HiRes / Fast variantsDiffer only in how many image tiles the document encoder processes, giving two quality-speed tradeoff points

An explanatory diagram made by METAL MEDIA, not a figure supplied by the paper's authors.

What they did

Problem: Visual document retrieval (VDR) encodes document pages as images to avoid OCR errors, but top models span 2 to 8 billion parameters, making indexing a million-page corpus cost tens of GPU-hours.
Method: A frozen 8B teacher (Qwen3-VL-Embedding-8B) precomputes target embeddings once; a 454M document student and a 70M query student are each trained independently to match those cached targets under a pointwise cosine alignment loss, needing no relevance labels or negative sampling.
Architecture: The document encoder splits page images into tiles, encodes them with an InternViT visual encoder, then re-contextualises them with a ModernBERT text backbone, while the query encoder is a lightweight text-only DistilBERT, matching the input asymmetry between image documents and text queries.
Results: DistilVDR-HiRes reaches 61.74 average NDCG@5 across the ViDoRe v1+v2+v3 suite (86.9% of the 8B teacher), leading every reproduced sub-1B baseline, while DistilVDR-Fast keeps 59.98 with a 3 times smaller visual-token budget; both store a million documents in a 15.6 times smaller index and index far faster than the strongest sub-1B multi-vector baseline.

Figure 2: Per-epoch train (blue) and validation (red) loss for the document students DistilVDR-HiRes (solid) and DistilVDR-Fast (dashed) on top, and the shared query student on bottom.

Table 1: Retrieval quality (NDCG@5) on ViDoRe v1, v2 and v3. “Type” is single vector or multi-vector with MaxSim.
Model	Params	Type	v1	v2	v3	Avg
Sub-1 B
SigLIP2-L	880 M	single	43.58	20.17	14.04	25.93
BiModernVBERT	250 M	single	37.40	10.88	5.52	17.93
colSmol-256M	256 M	multi	79.72	34.63	25.23	46.53
colSmol-500M	478 M	multi	82.42	43.09	33.52	53.01
ColModernVBERT	250 M	multi	76.76	33.18	17.45	42.46
SauerkrautLM-ColLFM2	451 M	multi	78.24	45.09	33.19	52.17
DistilVDR-Fast (ours)	524 M	single	81.34	54.95	43.66	59.98
DistilVDR-HiRes (ours)	524 M	single	82.81	55.34	47.07	61.74
Mid- to large-scale references
DSE-Qwen2	2.2 B	single	85.14	55.70	41.28	60.71
Qwen3-VL-Embedding-2B	2.1 B	single	84.30	65.25	49.98	66.51
ColPali v1.3	2.9 B	multi	84.21	54.72	42.04	60.32
Tomoro-ColQwen3-4B	4.4 B	multi	90.22	65.25	57.57	71.01
ColNomic-7B	7.0 B	multi	89.76	60.44	55.87	68.69
Tomoro-ColQwen3-8B	8.8 B	multi	90.61	65.00	59.00	71.54
Qwen3-VL-Embedding-8B (teacher)	8.1 B	single	87.31	69.76	56.07	71.05

Table 2: Gap decomposition (NDCG@5) for DistilVDR-HiRes. T = teacher, S = student. Bottom row is the full system.
Query × Doc	v1	v2	v3	Avg
T × T (oracle)	87.31	69.76	56.07	71.05
T × S	83.72	60.92	50.43	65.02
S × T	84.68	64.30	50.09	66.36
S × S (ours)	82.81	55.34	47.07	61.74

Table 3: Deployment efficiency on a single H200 GPU at fixed batch size B=8 in bf16. Each encoder runs under its best-supported flash-attention backend Dao et al. (2022); Dao (2024); per-baseline details are in Appendix G. Index size is per million documents. Avg NDCG@5 from Table 1 is repeated for reference. In each efficiency column we bold the column-best and underline the second-best.
Model	Params	Type	Query (ms)	Query	(ms)	Doc thpt (docs/s)	Doc thpt	(docs/s)
Query
(ms)
Doc thpt
(docs/s)
Peak VRAM
(GB)
Index
/ 1 M
Score 10 K
(ms)
Avg
NDCG@5
Sub-1 B
SigLIP2-L	880 M	single	113.8	28.22	1.98	4.1 GB	1.3	25.93
BiModernVBERT	250 M	single	7.4	2.49	7.47	3.1 GB	0.9	17.93
colSmol-256M	256 M	multi	23.6	2.58	4.49	256 GB	1 259	46.53
colSmol-500M	478 M	multi	22.6	2.82	4.97	256 GB	1 161	53.01
ColModernVBERT	250 M	multi	61.8	3.06	4.26	256 GB	1 238	42.46
SauerkrautLM-ColLFM2	451 M	multi	8.5	19.02	2.56	256 GB	1 330	52.17
DistilVDR-Fast (ours)	524 M	single	3.4	99.04	2.10	16.4 GB	9.6	59.98
DistilVDR-HiRes (ours)	524 M	single	3.4	36.82	3.07	16.4 GB	9.6	61.74
Mid- to large-scale references
DSE-Qwen2	2.2 B	single	167.4	17.17	6.31	6.1 GB	2.2	60.71
Qwen3-VL-Embedding-2B	2.1 B	single	14.5	8.53	7.03	8.2 GB	3.4	66.51
ColPali v1.3	2.9 B	multi	266.7	17.24	7.81	264 GB	1 158	60.32
Tomoro-ColQwen3-4B	4.4 B	multi	266.4	11.91	12.93	819 GB	3 187	71.01
ColNomic-7B	7.0 B	multi	542.6	9.96	17.88	256 GB	1 206	68.69
Tomoro-ColQwen3-8B	8.8 B	multi	499.3	9.20	21.76	819 GB	3 176	71.54
Qwen3-VL-Embedding-8B (teacher)	8.1 B	single	19.8	5.40	19.28	16.4 GB	9.4	71.05

Table 4: Document- and query-encoder ablations, NDCG@5. Blocks (a) and (b) are end-to-end student×student; block (c) is doc-side isolation (student documents against teacher queries); block (d) is query-side isolation (student queries against teacher documents), whose teacher×teacher ceiling is 71.05 average. Absolute values are therefore comparable within a block, not across blocks. † marks the deployed default.
Variant	v1	v2	v3	Avg
(a) Max tiles (end-to-end)
0 (no tiling)	77.57	51.29	41.00	56.62
2 (Fast)	81.34	54.95	43.66	59.98
6 (HiRes, default)	82.81	55.34	47.07	61.74
(b) Training-data scale (end-to-end)
25 % (300 K)	78.35	49.69	40.75	56.26
50 % (600 K)	80.97	54.44	44.58	60.00
75 % (900 K)	82.40	56.58	46.05	61.68
100 % (1.20 M, default)	82.81	55.34	47.07	61.74
(c) Doc output dim (doc-side isolation)
768-d (3.07 GB/1 M)	81.40	58.44	45.66	61.83
4096-d (16.4 GB/1 M)†	83.72	60.92	50.43	65.02
(d) Query backbone (query-side isolation)
DistilBERT-base (70 M)†	84.68	64.30	50.09	66.36
BERT-base (110 M)	81.81	58.52	44.55	61.63
ModernBERT-base (149 M)	85.36	65.40	51.43	67.40

Table 5: Contrastive-supervision ablation under Eq. 3. All rows are one-epoch joint refinements from the cosine-distilled 768-d sibling checkpoint; absolute NDCG@5 is below Section 4.2 because of the 768-d output.
Refinement objective	v1	v2	v3	Avg
γ=0 (cosine only)	79.90	53.23	42.22	58.45
+ 0.5ℒInfoNCE	80.47	52.08	42.44	58.33
+ 1.0ℒInfoNCE	80.48	51.80	42.41	58.23
+ 2.0ℒInfoNCE	80.25	51.31	42.30	57.95
+ 0.5ℒKL	80.32	52.31	42.30	58.31
+ 1.0ℒKL	80.20	52.55	42.34	58.36
+ 2.0ℒKL	80.31	52.54	42.44	58.43

Table 6: All 22 ViDoRe evaluation datasets. “Doc” is the document corpus language. “Query” is the languages in which queries are released. “Source” indicates how the queries were obtained: H = human-authored, L = LLM-generated, L+H = LLM-generated with expert review. The six languages of v3 are English, French, Spanish, German, Italian, and Portuguese.
Dataset	Ver.	Document domain	Doc	Query	Source
DocVQA	v1	Industrial documents	EN	EN	H
ArXivQA	v1	Scientific papers	EN	EN	H
InfoVQA	v1	Infographics	EN	EN	H
TatDQA	v1	Financial tables	EN	EN	H
TabFQuAD	v1	Tables in French PDFs	FR	FR	H
SyntheticDocQA-AI	v1	AI documents	EN	EN	L
SyntheticDocQA-Energy	v1	Energy sector reports	EN	EN	L
SyntheticDocQA-Gov.	v1	Government reports	EN	EN	L
SyntheticDocQA-Hlt.	v1	Healthcare documents	EN	EN	L
ShiftProject	v1	Environmental reports	FR	FR	L
ESG Reports	v2	ESG / sustainability	EN	EN/FR/ES/DE	L+H
Biomedical Lectures	v2	Biomedical slides	EN	EN/FR/ES/DE	L+H
Economics Reports	v2	Economics reports	EN	EN/FR/ES/DE	L+H
ESG Reports (Human)	v2	ESG / sustainability	EN	EN	H
Finance-EN	v3	US annual reports	EN	6 languages	L+H
Finance-FR	v3	French annual reports	FR	6 languages	L+H
Computer Science	v3	CS textbooks	EN	6 languages	L+H
HR	v3	EU HR reports	EN	6 languages	L+H
Energy	v3	French energy reports	FR	6 languages	L+H
Industrial	v3	USAF technical orders	EN	6 languages	L+H
Pharmaceutical	v3	FDA reports	EN	6 languages	L+H
Physics	v3	French physics lectures	FR	6 languages	L+H

Table 7: Training data composition. HuggingFace identifiers are hyperlinked; the organisation prefix is omitted in the second and third groups.
Source	Samples
Base mixture (711 K)
VisRAG-Ret-Train-Synthetic Yu et al. (2025)	234 K
VisRAG-Ret-Train-In-domain Yu et al. (2025)	94 K
colpali_train_set Faysse et al. (2025)	109 K
vdr-multilingual-train	275 K
Multi-domain supplement (Racineai, deduplicated)
racineai/VDR_* (14 sub-sources)	454 K
Finance supplement (31.7 K)
Sujet-Finance-Vision-10k	9.8 K
FinHNQue	21.9 K
Document encoder total	1.20 M
Query encoder mixture
NanoVDR query training set Liu et al. (2026)	1.49 M

Table 8: Training recipe under the cosine alignment objective (Eq. 1–2).
Hyperparameter	Doc enc.	Query enc.
Trainable parameters	454 M	70 M
Optimizer	AdamW	AdamW
Peak LR	1×10−3	5×10−4
LR schedule	one-cycle, 3% warmup
Effective batch	256	512
Epochs	3	15
Hardware	2× H200	2× H200

Findings

DistilVDR-HiRes reaches 61.74 average NDCG@5 across ViDoRe v1+v2+v3 (86.9% of the 8B teacher), beating the strongest reproduced sub-1B baseline colSmol-500M (53.01) by 8.73 points.
DistilVDR-Fast keeps 59.98 average NDCG@5 despite a 3 times smaller visual-token budget, still exceeding every reproduced sub-1B baseline.
On the high-resolution-sensitive v3 benchmark, HiRes scores 47.07, leading the next-best sub-1B baseline (33.52) by 13.55 points.
Both variants store one million documents in a 15.6 times smaller index than the strongest sub-1B multi-vector baseline (16.4 GB vs 256 GB) and index roughly an order of magnitude faster, with Fast reaching 99.04 docs/sec (about 18 times the teacher's throughput).
Both variants encode a query in 3.4 ms, faster than every other profiled system, and score 10,000 documents in 9.6 ms versus 1.1 to 3.2 seconds for multi-vector MaxSim baselines.

Where it can be used

Enterprise document search systems that need to index large volumes of scanned reports or PDFs as images for fast retrieval
Visual retrieval-augmented generation pipelines that need to preserve tables, figures, and layout when feeding retrieved pages to a language model
Deployments with limited GPU memory or serving budget looking for a compact substitute for multi-billion-parameter embedding models
Large-scale document archives that need to shrink index storage while retaining most of a strong teacher model's retrieval quality

Limits and open work

Every baseline compared is an official public release evaluated as-is, with different teachers, training data, and architectures, so the reported margin cannot be attributed to distillation alone.
No control experiment trains the same 524M architecture from scratch with contrastive learning and hard negatives on the same data, so distillation itself is not isolated from other factors.
Only one teacher model was used; the paper does not test how a weaker or stronger teacher, or a different model family, would change results.
DistilVDR still trails the strongest 4-8B multi-vector models by 7 to 10 points, with the largest gap on ViDoRe v3.
The query encoder is text-only and covers English plus five translated European languages, so image-conditioned queries and non-Latin scripts like Chinese, Japanese, or Arabic are out of scope, and the visual-tile budget is fixed regardless of page complexity.

Why it matters

Serving multi-billion-parameter retrieval models is expensive in GPU memory and indexing time, and this work shows a compact retriever can be distilled purely from a teacher's embeddings without any labeled relevance data. That gives practitioners a concrete recipe for building cheaper document-search or visual retrieval-augmented generation systems without training from scratch.

Terms in this paper

Visual document retrieval (VDR) · Retrieving relevant pages by encoding document images directly, instead of running OCR text extraction first
Cosine alignment loss · A training objective that pushes a student's output vector direction to match a fixed target vector's direction
Single-vector vs multi-vector retrieval · Representing a document as one vector (compact) versus many per-token vectors (larger storage, finer matching)
NDCG@5 · A ranking quality metric measuring how well the top 5 retrieved results match relevance, higher is better
MaxSim · A late-interaction scoring method in multi-vector retrieval that takes the best-matching token pair between query and document

Original abstract (English)

Visual document retrieval (VDR) is dominated by multi-billion-parameter models that are slow to index at full corpus scale and expensive to serve. Prior compression routes either train a smaller multi-vector encoder from scratch or distil only the query side; neither yields a compact single-vector retriever end-to-end. We present DistilVDR, a 524M end-to-end VDR system distilled bilaterally from a single 8B vision-language teacher under a pointwise cosine alignment loss. All supervision comes from the frozen teacher's embedding space, which was itself trained with relevance supervision, so the student objective needs no relevance labels, negative sampling, or contrastive term. We match VDR's text-query and image-document input asymmetry with an asymmetric encoder-only student that concentrates visual capacity on the document side and keeps the query side at 70M parameters. We release two variants that share the same encoders and training and differ only in the document encoder's visual-tile budget: DistilVDR-HiRes attains 61.74 average NDCG@5 on ViDoRe v1+v2+v3 (86.9% of the 8B teacher) and leads every reproduced sub-1B baseline on the high-resolution-sensitive v3 benchmark, while DistilVDR-Fast attains 59.98 at a 3 times smaller visual-token budget. Both variants store one million documents in a 15.6 times smaller index than the strongest sub-1B multi-vector baseline and index the corpus an order of magnitude faster. The code is available at https://github.com/Ryenhails/NanoVDR.

Authors · Zhuchenyang Liu

Read on arXiv

Latest papers

All papers →

Latest from METAL MEDIA

NCT 127 drop BLINGY, their first full album in two years

Figures: Zhuchenyang Liu et al., arXiv:2608.10636, CC BY 4.0