RynnWorld-Teleop：一種用於數字遙操作的動作條件世界模型

摘要

扩展机器人学习需要大量、多样化的轨迹数据，然而目前数据的采集受限于物理遥操作——每次演示都需要操作者在特定硬件和工作站上投入时间。为此，我们提出数字遥操作这一范式，通过用生成式世界模型替代真实机器人，将数据采集从物理约束中解耦出来。在该框架下，操作者的手部姿态流驱动一个以机器人为中心的生成式世界模型，基于单张参考图像合成高保真的自我中心视角视频。记录的姿态流作为无具身依赖性的动作标签，可通过标准重定向技术迁移至任意目标机器人，从而生成独立于物理硬件的完整状态-动作轨迹，用于模仿学习。我们将这一范式具体实现为RynnWorld-Teleop系统，该系统集成了深度感知的骨骼条件建模、渐进式人-机器人训练（基于视频扩散变换器）以及流式自回归蒸馏。该流水线将生成过程压缩为单次推理，在单张H100 GPU上实现40帧/秒以上的实时交互式生成。仅使用RynnWorld-Teleop生成的数据训练的策略，能够在多种灵巧及双臂协作任务中实现有效的零样本仿真到现实迁移。此外，用我们的数字遥操作数据增强真实世界数据集，能够持续提升任务成功率，证明RynnWorld-Teleop可作为面向下一代机器人智能体的高保真、可扩展的数据引擎。

English

Scaling robot learning requires massive, diverse trajectory data, yet collection is currently bottlenecked by physical teleoperation, where every demonstration binds operator time to specific hardware and workspaces. We introduce digital teleoperation, a paradigm that decouples data collection from physical constraints by replacing the real robot with a generative world model. In this framework, an operator's hand-pose stream drives a robot-centric generative world model to synthesize high-fidelity egocentric videos from a single reference image. The recorded pose stream serves as an embodiment-agnostic action label transferable to any target robot via standard retargeting, yielding complete state-action trajectories for imitation learning independent of physical hardware. We instantiate this paradigm in RynnWorld-Teleop, a system that integrates depth-aware skeletal conditioning, progressive human-to-robot training on a video Diffusion Transformer, and streaming autoregressive distillation. This pipeline compresses the generative process into a single-pass inference, enabling 40+ FPS, real-time interactive generation on a single H100 GPU. Policies trained exclusively on RynnWorld-Teleop-generated data achieve effective zero-shot Sim2Real transfer across dexterous and diverse bimanual tasks. Moreover, augmenting real-world datasets with our digitally teleoperated data consistently improves success rates, demonstrating that RynnWorld-Teleop serves as a high-fidelity, scalable data engine for the next generation of robotic agents.