EgoSteer：基於自我中心視頻的可操控靈巧操作全棧系統

摘要

操控性是通用型機器人策略的核心能力，但在靈巧手系統中仍嚴重缺乏，原因在於缺乏大規模、語言對齊且動作精準的示範資料。為解決此瓶頸，我們提出一套全端系統，可從第一人稱人類影片擴展靈巧VLA預訓練，並支援資料高效的真實機器人後訓練。該系統整合了EgoSmith資料管線，可將野外第一人稱影片整理成9,600小時的高品質預訓練資料，其吞吐量比先前最佳技術高出9倍，準確性也更高；還有統一的機器人堆疊，用於遠端操作與人機協作修正；以及EgoSteer，這是一個基於優化基礎設施訓練的世界模型增強型VLA。人類資料預訓練賦予EgoSteer語言引導的操作先驗，再透過機器人後訓練紮根，並經由DAgger改進強化。實證結果顯示，EgoSteer能在40多種多樣任務中穩健執行自由形式指令，展現故障恢復、靈巧性與泛化能力。預訓練模型還能少樣本適應複雜的長時程任務，包括在兩種實體上摺疊紙盒，成功率超過75%。我們已在https://egosteer.github.io/ 開源系統、資料與模型。

English

Steerability is a defining capability of generalist robot policies, yet remains largely absent in dexterous-hand systems for lack of large-scale, language-aligned, and action-accurate demonstration data. To address this bottleneck, we present a full-stack system that scales dexterous VLA pre-training from egocentric human videos and enables data-efficient real-robot post-training. It integrates EgoSmith, a data pipeline that curates in-the-wild egocentric videos into 9.6K hours of high-quality pre-training data with 9x higher throughput and better accuracy than prior SOTA; a unified robot stack for teleoperation and human-in-the-loop correction; and EgoSteer, a world-model-enhanced VLA trained on optimized infrastructure. Human-data pre-training equips EgoSteer with language-guided manipulation priors, which are grounded through robot post-training and improved by DAgger refinement. Empirically, EgoSteer robustly executes free-form instructions across 40+ diverse tasks, demonstrating failure recovery, dexterity, and generalization. The pre-trained model also few-shot adapts to complex long-horizon tasks, including box folding, on two embodiments with 75+% success. We open-source the system, data, and model at https://egosteer.github.io/.