面向具身智能的专家混合视频预训练规模化

摘要

尽管机器人控制领域近期取得了进展，但视频生成模型因主要聚焦于内容创作而面临领域不匹配的问题。例如，其设计本质上优先考虑视觉保真度和创造性，而非计算效率与物理真实性。在本研究中，我们提出LingBot-Video——一种专为具身智能设计的基于DiT（扩散Transformer）的视频预训练范式。从架构角度，我们采用混合专家模型（MoE）而非密集框架，以更好平衡建模能力与推理效率，并成功实现从零开始的规模化扩展。从数据角度，我们构建了数据剖析引擎，通过整合大量机器人导向的素材（涵盖操作、导航及第一人称视角）来增强标准互联网视频，从而赋予基础模型对动作与世界动态的内在理解。从训练角度，我们开发了多维度奖励系统，在美学标准、指令遵循和运动一致性等常规准则之外，进一步强化对物理合理性及任务完成度的对齐。全面的评估验证了其作为视频基础模型的性能与效率。我们贡献LingBot-Video作为社区首个大规模开源MoE视频基础模型，旨在开创性地弥合数字创造力与物理执行之间的鸿沟。

English

Despite the recent promise in robot control, video generative models suffer from a domain mismatch due to their primary focus on content creation. For example, their design inherently prioritizes visual fidelity and creativity over computational efficiency and physical realism. In this work, we present LingBot-Video, a DiT-based video pretraining paradigm specifically tailored for embodied intelligence. From the architecture perspective, we adopt the Mixture-of-Experts (MoE), instead of dense, framework to achieve a better trade-off between modeling capacity and inference efficiency, and manage to scale it up from scratch. From the data perspective, we construct a data profiling engine that augments standard internet videos with extensive robot-oriented footage, encompassing manipulation, navigation, and egocentric perspectives, to equip the base model with an intrinsic understanding of actions and world dynamics. From the training perspective, we develop a multi-dimensional reward system to enforce the alignment regarding physical rationality and task completion, going beyond standard criteria such as aesthetics, prompt-following, and motion consistency. Comprehensive evaluations validate its performance and efficiency as a video foundation model. We contribute LingBot-Video as the inaugural large-scale, open-source MoE video foundation model to the community, in a pioneering effort to bridge digital creativity and physical actuation.