Wan-Streamer v0.1：端到端即時互動式基礎模型

摘要

我們提出了 Wan-Streamer，這是一個原生串流、端到端的互動基礎模型，從底層專為即時、低延遲、全雙工的音視頻互動而設計。Wan-Streamer 在單一 Transformer 中無縫地將語言、音頻和視頻同時作為輸入和輸出進行建模，其中序列表示為交錯的視覺、音頻和文本輸入令牌，以及視覺、音頻和文本輸出令牌，並透過區塊因果注意力（block-causal attention）進行協調以實現增量串流。與依賴獨立 VAD、ASR、語言、TTS、音頻驅動動畫或視頻生成模組的串聯互動系統不同，Wan-Streamer 不依賴外部語言、語音、虛擬角色或視頻生成模組：感知、推理、生成、回應時序、輪次管理以及跨模態同步皆在一個統一模型中共同學習，從而減少管線延遲和錯誤累積。為了支援自然的音視頻回應能力，我們圍繞串流性重新設計了整個技術堆疊，包括因果編碼器、因果解碼器、區塊因果注意力以及低延遲的多模態令牌排程，使得串流單元可短至 160 毫秒（25 fps）。Wan-Streamer 實現了約 200 毫秒的模型端回應延遲，並在結合 350 毫秒的雙向網路延遲後，總互動延遲約為 550 毫秒，支援亞秒級的雙工音視頻通訊。這些成果使 Wan-Streamer 成為一個用於低延遲串流互動的統一、端到端、多模態互動基礎模型。

English

We present Wan-Streamer, a native-streaming, end-to-end interactive foundation model designed from the ground up for real-time, low-latency, full-duplex audio-visual interaction. Wan-Streamer seamlessly models language, audio, and video as both input and output within a single Transformer, where the sequence is represented as interleaved visual, audio, and text input tokens together with visual, audio, and text output tokens, coordinated by block-causal attention for incremental streaming. Unlike cascaded interactive systems that rely on separate VAD, ASR, language, TTS, audio-driven animation, or video-generation modules, Wan-Streamer does not rely on external language, speech, avatar, or video-generation modules: perception, reasoning, generation, response timing, turn management, and cross-modal synchronization are learned jointly within one unified model, reducing pipeline latency and error accumulation. To support natural audio-visual responsiveness, we redesign the entire stack around streamability, including causal encoders, causal decoders, block-causal attention, and low-latency multimodal token scheduling, enabling streaming units as short as 160 ms at 25 fps. Wan-Streamer achieves approximately 200 ms model-side response latency and approximately 550 ms total interaction latency when combined with 350 ms bidirectional network latency, supporting sub-second duplex audio-visual communication. These results position Wan-Streamer as a unified, end-to-end, multimodal interactive foundation model for low-latency streaming interaction.