LeapTalk：突破說話頭像生成中的延遲-品質權衡

摘要

長篇即時說話人頭生成仍因延遲與品質之間的權衡而充滿挑戰：低效率的多步擴散無法支援串流生成，而即時自迴歸方法則飽受誤差累積與身分漂移之苦。為了解決此缺陷，我們提出 LeapTalk，一個新穎的框架，能以單次前向步驟實現穩定且即時的說話人頭生成，並可擴展至任意長度的影片。我們方法的核心在於單步橋接蒸餾方案。一方面，有別於傳統的雜訊到資料範式，我們引入基於布朗橋的資料到資料傳輸公式。以持久參考為錨定，此策略能有效減緩身分漂移並增強長期時間穩定性。另一方面，為了實現從預訓練擴散教師模型到學生橋接模型的平滑知識遷移，我們探索了一種異質蒸餾框架，搭配 SNR 對齊的時間變換 Φ(τ)，以彌合兩個模型之間的功能差異。此外，我們提出了音訊驅動的無分類器引導機制，以在極端步數縮減下維持精細的唇形同步。大量實驗證明，我們的方法僅需 1 步即可生成高保真且時間一致的影片，幀率可達 200 FPS，在效率與穩定性上均顯著優於現有方法。專案頁面：https://zhangrongxiang.github.io/leaptalk-page/

English

Long-form and real-time talking-head generation remains challenging due to a latency-quality trade-off: inefficient multi-step diffusion prohibits streaming generation, whereas real-time autoregressive approaches suffer from error accumulation and identity drift. To address this drawback, we propose LeapTalk, a novel framework that achieves stable and real-time talking-head generation with a single forward step, scaling to arbitrarily long videos. At the heart of our approach lies a single-step bridge distillation scheme. On the one hand, departing from the conventional noise-to-data paradigm, we introduce a data-to-data transport formulation based on a Brownian bridge. Anchored by a persistent reference, this strategy effectively mitigates identity drift and enhances long-term temporal stability. On the other hand, to enable smooth knowledge transfer from a pre-trained diffusion teacher to the student bridge model, we explore a heterogeneous distillation framework with an SNR-aligned time transformation Φ(τ), which bridges the functional discrepancy between the two models. Moreover, we propose an audio-driven classifier-free guidance mechanism to maintain fine-grained lip synchronization under extreme step reduction. Extensive experiments demonstrate that our method achieves high-fidelity and temporally consistent video generation with only 1 step at up to 200 FPS, significantly outperforming existing approaches in both efficiency and stability. Project Page: https://zhangrongxiang.github.io/leaptalk-page/