DICS：探索數據內在一致性以進行視覺指令選擇

摘要

視覺指令微調對於提升視覺語言模型（Vision-Language Models, VLMs）的視覺語言對齊與指令跟隨能力至關重要。然而，在固定比例限制下，從快速擴張的資料集中識別最佳子集仍是一項重大瓶頸。儘管現有方法大多依賴分佈多樣性或啟發式過濾，它們往往忽略了單一樣本內部的內在一致性。為填補此一缺口，我們提出資料內在一致性（Data Intrinsic Consistency, DIC），這是一種自評分指標，旨在量化樣本層級的元件間一致性。DIC 由兩個模組組成：視覺資訊一致性（Visual Information Consistency, VIC），評估視覺內容與指令之間的對齊程度；以及回應資訊一致性（Response Information Consistency, RIC），評估回應相對於指令的一致性。基於 DIC，我們進一步提出資料內在一致性選擇（Data Intrinsic Consistency Selection, DICS），這是一種自適應資料選擇方法，可在不同資料預算下優化高樣本內一致性與整體分佈多樣性之間的取捨。大量實驗表明，DICS 在多種資料集規模與模型架構下均持續優於最先進的方法，僅使用 LLaVA-1.5-665K 資料的 25% 即超越了全資料集微調的表現。我們進一步建構了 DICS-6M，這是一個包含 600 萬個樣本的多模態指令語料庫，實現了迄今最大規模的視覺指令選擇研究；值得注意的是，DICS 在不到其官方報告訓練資料 25% 的條件下，達到了 InternVL3-8B-Instruct 官方版本的 94.52% 效能。程式碼可見於 https://github.com/cqu-student/DICS

English

Visual instruction tuning is crucial for advancing the vision-language alignment and instruction-following capabilities of Vision-Language Models (VLMs). However, identifying optimal subsets under a fixed ratio constraint from rapidly expanding datasets remains a significant bottleneck. While existing methods largely depend on distribution diversity or heuristic filtering, they often overlook the internal coherence within individual samples. To bridge this gap, we propose Data Intrinsic Consistency (DIC), a self-scoring metric designed to quantify the sample-level inter-component consistency. DIC consists of two modules: Visual Information Consistency (VIC), evaluating the alignment between visual content and instructions, and Response Information Consistency (RIC), assessing response coherence relative to the instruction. Building upon DIC, we introduce Data Intrinsic Consistency Selection (DICS), an adaptive data selection method that optimizes the trade-off between high intra-sample consistency and global distributional diversity under varying data budgets. Extensive experiments demonstrate that DICS consistently outperforms state-of-the-art methods across diverse dataset scales and model architectures, surpassing full-dataset fine-tuning while using only 25% of the LLaVA-1.5-665K data. We further curate DICS-6M, a 6M-sample multi-modal instruction corpus that enables the largest-scale visual instruction selection study to date; remarkably, DICS reaches 94.52\% of the official InternVL3-8B-Instruct performance using less than 25\% of its reported training data. Code can be seen at https://github.com/cqu-student/DICS