自监督视觉在策略蒸馏

摘要

视觉在策略蒸馏在很大程度上依赖于信息丰富的师生非对称性，这种非对称性要么来自更大、更强的教师模型，要么来自特权监督，例如参考答案或感兴趣区域的真值。这引出了一个基本问题：当没有任何特权信息可用时，信息丰富的非对称性从何而来？我们通过反转非对称性的来源来回答这一问题。我们不是向教师添加特权信息，而是从学生中减去信息。这种非对称性无需真值标注、奖励或独立的更强教师模型，就能免费产生与能够获取学生无法获得信息的教师相同的有效学习信号。基于这一原则，我们提出了自监督视觉在策略蒸馏（S^2VOPD），这是一种简单而有效的方法，通过非对称增强视图构建在策略学习信号。S^2VOPD 将在策略的、以原始图像为条件的教师分布蒸馏到以同一图像的强增强视图为条件的学生分布中。我们系统地探索了视觉增强的广泛设计空间，并发现：（1）非对称性至关重要：所有四类增强都能提升性能，而对称自蒸馏则会降低性能；（2）强度很重要：性能在中等强度时达到峰值；（3）差距必须保持任务一致：完全移除与问题相关证据的增强会引发很大但无信息的差异。在六个细粒度感知基准测试中，S^2VOPD 将 Qwen3.5-4B 从 70.7% 提升至 77.4%，超过所有对比的开源模型，包括高达 235B 参数的 Qwen3-VL，并超越了 GPT-5.4。在保持训练数据相同的情况下，它恢复了使用特权信息的方法所取得改进的 96%。网站位于 https://williamium3000.github.io/s2vopd

English

Visual on-policy distillation relies heavily on an informative teacher-student asymmetry, through either a larger, stronger teacher or privileged supervision, such as reference answers or ground-truth regions of interest. This raises a fundamental question: where can informative asymmetry come from when nothing privileged is available? We answer this by inverting where the asymmetry comes from. Rather than adding privileged information to the teacher, we subtract information from the student. This asymmetry creates the same effective learning signal for free as a teacher with access to information unavailable to the student, without ground-truth annotations, rewards, or a separate stronger teacher model. Building on this principle, we introduce Self-Supervised Visual On-Policy Distillation (S^2VOPD), a simple yet effective method that constructs on-policy learning signals from asymmetric augmented views. S^2VOPD distills the teacher's distribution conditioned on the original image on-policy into the student distribution conditioned on a strongly augmented view of the same image. We systematically explore a broad design space of visual augmentations and uncover that (1) asymmetry matters: all four augmentation families improve performance, while symmetric self-distillation degrades it; (2) strength matters: performance peaks at a moderate strength; and (3) the gap must remain task-consistent: augmentations that completely remove the question-relevant evidence can induce large but uninformative discrepancies. Across six fine-grained perception benchmarks, S^2VOPD improves Qwen3.5-4B from 70.7% to 77.4%, above all open-source models compared, up to Qwen3-VL at 235B, and surpasses GPT-5.4. While holding training data the same, it recovers 96% of the improvement achieved by methods with privileged information. Website is at https://williamium3000.github.io/s2vopd