CogniRoute：在全模态模型中学习路由社会证据

摘要

全模态模型能够处理视频、音频和文本，但统一接入多种模态并不能保证模型使用正确的证据。这一差距在社交视频问答中尤为突出——答案可能取决于某个手势、语调、时间线索，或者所言与所视之间的不匹配。我们提出CogniRoute，一个面向社交全模态推理的模式引导混合专家框架。CogniRoute采用仅用于训练阶段的认知模式，通过跨模态关系、推理需求和时间范围对每个样本进行分解，并在监督微调过程中将全局路由特征与该结构对齐。我们进一步引入路由感知强化学习，利用面向答案正确性、模态一致推理和认知时间锚定性的奖励，联合优化令牌生成与专家分配。为支持训练与评估，我们构建了OmniSocialBench，这是一个诊断性社交视频问答资源，包含11.8万条结构化训练样本、基于推理轨迹、模式标签、时间证据片段以及人工验证的评估划分。CogniRoute在OmniSocialBench上达到59.38%的平均准确率，较最强专有基线提升15.33个百分点，较最强开源全模态基线提升26.77个百分点，其中在需要音视频协同、冲突消解以及时间锚定社交推理的问题上增益最大。

English

Omni-modal models can ingest video, audio, and text, but unified access to multiple modalities does not guarantee that a model uses the right evidence. This gap is especially pronounced in social video question answering, where the answer may hinge on a gesture, vocal tone, temporal cue, or mismatch between what is said and what is visually expressed. We introduce CogniRoute, a schema-guided Mixture-of-Experts framework for social omni reasoning. CogniRoute uses a training-only cognitive schema that factorizes each example by cross-modal relation, reasoning demand, and temporal scope, and aligns global routing signatures with this structure during supervised fine-tuning. We further introduce route-aware reinforcement learning, which jointly optimizes token generation and expert allocation using rewards for answer correctness, modality-consistent reasoning, and cognitive temporal grounding. To support training and evaluation, we construct OmniSocialBench, a diagnostic social video QA resource with 118K structured training examples, grounded reasoning traces, schema labels, temporal evidence spans, and a manually verified evaluation split. CogniRoute achieves 59.38\% average accuracy on OmniSocialBench, improving over the strongest proprietary baseline by 15.33 percentage points and the strongest open-source omni baseline by 26.77 points, with the largest gains on questions requiring audio-visual coordination, conflict resolution, and temporally grounded social inference.