DistilVDR：一种基于双学生蒸馏的紧凑型端到端视觉文档检索器

摘要

视觉文档检索（VDR）目前由数十亿参数模型主导，这些模型在全语料库规模上索引速度慢且服务成本高昂。此前的压缩路线要么从头训练一个更小的多向量编码器，要么只对查询侧进行蒸馏，两者都无法端到端地得到一个紧凑的单向量检索器。我们提出了 DistilVDR，一个 524M 参数的端到端 VDR 系统，在逐点余弦对齐损失下，从单个 8B 视觉-语言教师模型中双侧蒸馏而来。所有监督信号均来自冻结教师模型的嵌入空间，而该教师模型本身在训练时使用了相关性监督，因此学生目标函数无需相关性标签、负采样或对比项。我们用一个不对称的仅编码器学生模型来匹配 VDR 的文本查询—图像文档输入不对称性，将视觉能力集中在文档侧，查询侧参数保持为 70M。我们发布了两个变体，它们共享相同的编码器和训练过程，仅文档编码器的视觉令牌预算不同：DistilVDR-HiRes 在 ViDoRe v1+v2+v3 上取得 61.74 的平均 NDCG@5（达到 8B 教师模型的 86.9%），并在对高分辨率敏感的 v3 基准上领先所有已复现的 sub-1B 基线；DistilVDR-Fast 在视觉令牌预算仅为前者三分之一的条件下取得 59.98。两个变体都能将一百万文档存储在比最强的 sub-1B 多向量基线小 15.6 倍的索引中，并以快一个数量级的速度完成语料库索引。代码可在 https://github.com/Ryenhails/NanoVDR 获取。

English

Visual document retrieval (VDR) is dominated by multi-billion-parameter models that are slow to index at full corpus scale and expensive to serve. Prior compression routes either train a smaller multi-vector encoder from scratch or distil only the query side; neither yields a compact single-vector retriever end-to-end. We present DistilVDR, a 524M end-to-end VDR system distilled bilaterally from a single 8B vision-language teacher under a pointwise cosine alignment loss. All supervision comes from the frozen teacher's embedding space, which was itself trained with relevance supervision, so the student objective needs no relevance labels, negative sampling, or contrastive term. We match VDR's text-query and image-document input asymmetry with an asymmetric encoder-only student that concentrates visual capacity on the document side and keeps the query side at 70M parameters. We release two variants that share the same encoders and training and differ only in the document encoder's visual-tile budget: DistilVDR-HiRes attains 61.74 average NDCG@5 on ViDoRe v1+v2+v3 (86.9% of the 8B teacher) and leads every reproduced sub-1B baseline on the high-resolution-sensitive v3 benchmark, while DistilVDR-Fast attains 59.98 at a 3 times smaller visual-token budget. Both variants store one million documents in a 15.6 times smaller index than the strongest sub-1B multi-vector baseline and index the corpus an order of magnitude faster. The code is available at https://github.com/Ryenhails/NanoVDR.