LiveAnimate：即時穩定長篇流式人體動畫

摘要

姿態驅動的人體動畫旨在從單張參考圖像與一條驅動姿態串流中，合成目標人物的影片。即時生成對於直播、遠距臨場與虛擬化身等互動應用至關重要；然而，基於擴散的系統生成一段剪輯需要數分鐘到數小時，無法支援即時互動。我們提出 LiveAnimate，據我們所知，這是第一個在十億參數量級上結合即時串流與穩定長序列生成的動畫系統，其核心為一個擁有 140 億參數的影片擴散 Transformer（DiT）。我們的兩階段訓練流程首先透過參考錨定教師強制適應（Reference-Anchored Teacher-Forcing Adaptation），將預訓練的雙向 DiT 改造為區塊因果自迴歸生成器；接著透過區塊式自強制蒸餾（Block-wise Self-Forcing Distillation），將取樣預算降至三步。為了在長串流中保持外觀，我們引入姿態檢索匯集注意力（Pose-Retrieval Sink Attention, PR-Sink）——一種有界 KV 快取機制，結合了永久錨定第一個生成區塊的靜態匯集（Static Sink）、保存姿態檢索歷史區塊的動態匯集（Dynamic Sink），以及一個三槽滾動視窗（Rolling Window）。當某個姿態再次出現時，PR-Sink 能還原相關的外觀上下文，而無需保留整個序列，因此記憶體使用量與每區塊延遲皆不隨串流時長增長。結合 Ulysses 序列平行與運算子融合，這些設計在兩張 NVIDIA H100 GPU 上實現了 19.63 FPS 的串流推論。在三分鐘的基準測試中，LiveAnimate 從前 30 秒到最後一分鐘幾乎維持恆定的感知品質與身份一致性，而既有系統在相同測試中要麼大幅衰退，要麼需要數小時的離線計算。這些結果為互動式全身動畫在品質、延遲與時長方面樹立了新的效能基準。

English

Pose-driven human animation synthesizes a video of a target person from a single reference image and a driving pose stream. Real-time generation is essential for interactive applications such as live streaming, telepresence, and virtual avatars, yet diffusion-based systems require minutes to hours per clip, precluding responsive interaction. We present LiveAnimate, to our knowledge the first animation system to combine real-time streaming with stable long-form generation at billion scale, built on a 14B-parameter video Diffusion Transformer (DiT). A two-stage training pipeline first adapts a pretrained bidirectional DiT into a block-causal autoregressive generator through Reference-Anchored Teacher-Forcing Adaptation, and then reduces the sampling budget to three steps through Block-wise Self-Forcing Distillation. To preserve appearance over extended streams, we introduce Pose-Retrieval Sink Attention (PR-Sink), a bounded KV-cache mechanism combining a Static Sink that permanently anchors the first generated block, a Dynamic Sink that holds a pose-retrieved historical block, and a three-slot Rolling Window. When a pose recurs, PR-Sink restores the relevant appearance context without retaining the entire sequence, so memory and per-block latency remain constant regardless of stream duration. Together with Ulysses sequence parallelism and operator fusion, these designs enable 19.63\,FPS streaming inference on two NVIDIA H100 GPUs. On a three-minute benchmark, LiveAnimate maintains nearly constant perceptual quality and identity from the first 30 seconds to the final minute, while prior systems degrade substantially or require hours of offline computation for the same rollout. These results establish a new operating point in quality, latency, and duration for interactive full-body animation.