HandX：扩展双手动作与交互生成技术

摘要

尽管人体运动合成技术发展迅猛，但真实的手部运动与双手交互研究仍显不足。现有全身模型往往忽略驱动灵巧行为的关键细节——手指关节活动、接触时序及双手协调等精细特征，且当前资源缺乏能捕捉细微手指动态与协作的高保真双手运动序列。为填补这一空白，我们提出HandX框架，构建涵盖数据、标注与评估的统一基础。我们整合并筛选现有数据集以提升质量，同时采集新的动作捕捉数据集，重点关注具有精细手指动态的典型双手交互场景。为实现可扩展标注，我们采用解耦策略：先提取代表性运动特征（如接触事件与手指屈伸），再借助大语言模型的推理能力生成与特征对齐的细粒度语义描述。基于所得数据与标注，我们以多模态条件生成为背景，对扩散模型与自回归模型进行基准测试。实验证明，结合我们新提出的手部专项评估指标，该方法能生成高质量的灵巧运动。我们进一步观察到明显的规模效应：使用更高质量的大规模数据集训练的大型模型，能产生语义更连贯的双手运动。本数据集已开源以支持后续研究。

English

Synthesizing human motion has advanced rapidly, yet realistic hand motion and bimanual interaction remain underexplored. Whole-body models often miss the fine-grained cues that drive dexterous behavior, finger articulation, contact timing, and inter-hand coordination, and existing resources lack high-fidelity bimanual sequences that capture nuanced finger dynamics and collaboration. To fill this gap, we present HandX, a unified foundation spanning data, annotation, and evaluation. We consolidate and filter existing datasets for quality, and collect a new motion-capture dataset targeting underrepresented bimanual interactions with detailed finger dynamics. For scalable annotation, we introduce a decoupled strategy that extracts representative motion features, e.g., contact events and finger flexion, and then leverages reasoning from large language models to produce fine-grained, semantically rich descriptions aligned with these features. Building on the resulting data and annotations, we benchmark diffusion and autoregressive models with versatile conditioning modes. Experiments demonstrate high-quality dexterous motion generation, supported by our newly proposed hand-focused metrics. We further observe clear scaling trends: larger models trained on larger, higher-quality datasets produce more semantically coherent bimanual motion. Our dataset is released to support future research.

HandX：扩展双手动作与交互生成技术

HandX: Scaling Bimanual Motion and Interaction Generation

摘要

Support