LeapTalk：突破说话头生成中的延迟-质量权衡

摘要

长时且实时的说话头生成仍然面临延迟-质量权衡的挑战：低效的多步扩散方法无法支持流式生成，而实时的自回归方法则饱受误差累积和身份漂移之苦。为解决这一缺陷，我们提出LeapTalk，一种新颖的框架，通过单前向步骤实现稳定且实时的说话头生成，并可扩展至任意长度的视频。我们方法的核心在于单步桥接蒸馏方案。一方面，我们摒弃传统的噪声到数据范式，引入基于布朗桥的数据到数据传输公式。通过持久参考的锚定，该策略有效缓解身份漂移并增强长期时间稳定性。另一方面，为促进从预训练扩散教师模型到学生桥接模型的知识平滑迁移，我们探索了一种异构蒸馏框架，结合信噪比对齐的时间变换Φ(τ)，从而弥合两个模型之间的功能差异。此外，我们提出一种音频驱动的无分类器引导机制，以在极端步数缩减下保持精细的唇形同步。大量实验表明，我们的方法仅需1步即可生成高保真且时间一致的视频，帧率高达200 FPS，在效率和稳定性方面均显著优于现有方法。项目页面：https://zhangrongxiang.github.io/leaptalk-page/

English

Long-form and real-time talking-head generation remains challenging due to a latency-quality trade-off: inefficient multi-step diffusion prohibits streaming generation, whereas real-time autoregressive approaches suffer from error accumulation and identity drift. To address this drawback, we propose LeapTalk, a novel framework that achieves stable and real-time talking-head generation with a single forward step, scaling to arbitrarily long videos. At the heart of our approach lies a single-step bridge distillation scheme. On the one hand, departing from the conventional noise-to-data paradigm, we introduce a data-to-data transport formulation based on a Brownian bridge. Anchored by a persistent reference, this strategy effectively mitigates identity drift and enhances long-term temporal stability. On the other hand, to enable smooth knowledge transfer from a pre-trained diffusion teacher to the student bridge model, we explore a heterogeneous distillation framework with an SNR-aligned time transformation Φ(τ), which bridges the functional discrepancy between the two models. Moreover, we propose an audio-driven classifier-free guidance mechanism to maintain fine-grained lip synchronization under extreme step reduction. Extensive experiments demonstrate that our method achieves high-fidelity and temporally consistent video generation with only 1 step at up to 200 FPS, significantly outperforming existing approaches in both efficiency and stability. Project Page: https://zhangrongxiang.github.io/leaptalk-page/