Super Star：迈向数字人的流式实时交互智能体

摘要

现有的共语手势生成方法主要是在离线设置下研究的，即从完整的语音片段中合成手势。然而，真实场景中的交互式数字人需要在严格的延迟约束下，仅利用当前可用的响应音频，在线生成与语音同步的手势。因此，先前的方法不适合实时交互，因为它们要么依赖未来语音信息，要么产生显著的推理延迟。本文针对交互式数字人定义了在线共语手势生成问题，并提出了一种实时交互框架，该框架将流式语音响应模块与在线手势生成模块相耦合。具体而言，手势生成器被设计为一种因果多模态自回归模型，从流式响应语音和运动历史中预测身体运动，从而在无需访问未来语音的情况下实现低延迟且与语音对齐的手势合成。为支持这一设置，我们进一步提出了一种针对虚拟伴侣场景定制的离线数据合成流水线，该流水线利用主题感知和情感感知的主体语料构建多样化的人机对话，然后基于智能体响应生成共语手势。此外，为弥合离线数据构建与在线部署之间的差距，我们建立了一个自演进训练循环，将在线交互过程中收集的用户反馈纳入数据生成流程，从而实现对用户偏好的持续适应。大量实验表明，与现有竞争基线相比，我们的框架在延迟-质量权衡、语音-动作同步性和用户偏好方面均取得了更优的结果。项目页面：https://super-star-2026.github.io/

English

Existing co-speech gesture generation methods are predominantly studied in offline settings, where gestures are synthesized from complete speech segments. However, interactive digital humans in real-world scenarios are required to generate speech-synchronous gestures online, using only currently available response audio under strict latency constraints. As a result, prior methods are unsuitable for real-time interaction, as they either rely on future speech information or incur substantial inference delay. In this paper, we formulate online co-speech gesture generation for interactive digital humans and propose a real-time interactive framework that couples a streaming speech response module with an online gesture generation module. Specifically, the gesture generator is designed as a causal multimodal autoregressive model that predicts body motion from streaming response speech and motion history, enabling low-latency and speech-aligned gesture synthesis without access to future speech. To support this setting, we further propose an offline data synthesis pipeline tailored to virtual companion scenarios, which leverages topic- and emotion-aware subject corpora to construct diverse human-agent dialogues and then generates co-speech gestures conditioned on the agent responses. Moreover, to bridge the gap between offline data construction and online deployment, we establish a self-evolving training loop by incorporating user feedback collected during online interaction into the data generation process, enabling continual adaptation to user preferences. Extensive experiments demonstrate that our framework achieves superior better latency-quality trade-off, stronger speech-motion synchronization, and higher user preference than competitive existing baselines. Project Page: https://super-star-2026.github.io/