CogniRoute：在全模态模型中学习路由社会证据

摘要

全模态模型能处理视频、音频和文本，但统一接入多种模态并不能保证模型会使用正确的证据。这一差距在社交媒体视频问答中尤为突出，因为答案可能取决于某个手势、语调、时间线索，或者言语与视觉表达之间的不一致。我们提出了 CogniRoute，一种基于模式引导的混合专家框架，用于社交全模态推理。CogniRoute 使用仅在训练时应用的认知模式，将每个示例按跨模态关系、推理需求和时间范围进行分解，并在监督微调过程中将全局路由特征与此结构对齐。我们进一步引入了路径感知强化学习，通过针对答案正确性、模态一致性推理以及认知时间定位的奖励，联合优化生成标记与专家分配。为支持训练和评估，我们构建了 OmniSocialBench，一个包含11.8万条结构化训练样本、有依据的推理轨迹、模式标签、时间证据跨度以及经过人工验证的评估子集的诊断性社交视频问答资源。CogniRoute 在 OmniSocialBench 上实现了59.38%的平均准确率，比最强的闭源基线高出15.33个百分点，比最强的开源全模态基线高出26.77个百分点，其中在需要视听协调、冲突解决以及基于时间的社交推理问题上提升最为显著。

English

Omni-modal models can ingest video, audio, and text, but unified access to multiple modalities does not guarantee that a model uses the right evidence. This gap is especially pronounced in social video question answering, where the answer may hinge on a gesture, vocal tone, temporal cue, or mismatch between what is said and what is visually expressed. We introduce CogniRoute, a schema-guided Mixture-of-Experts framework for social omni reasoning. CogniRoute uses a training-only cognitive schema that factorizes each example by cross-modal relation, reasoning demand, and temporal scope, and aligns global routing signatures with this structure during supervised fine-tuning. We further introduce route-aware reinforcement learning, which jointly optimizes token generation and expert allocation using rewards for answer correctness, modality-consistent reasoning, and cognitive temporal grounding. To support training and evaluation, we construct OmniSocialBench, a diagnostic social video QA resource with 118K structured training examples, grounded reasoning traces, schema labels, temporal evidence spans, and a manually verified evaluation split. CogniRoute achieves 59.38\% average accuracy on OmniSocialBench, improving over the strongest proprietary baseline by 15.33 percentage points and the strongest open-source omni baseline by 26.77 points, with the largest gains on questions requiring audio-visual coordination, conflict resolution, and temporally grounded social inference.