原生音视频对齐生成

摘要

联合音视频生成旨在合成时间同步且语义一致的视听内容。然而，现有开源方法主要依赖两种设计：一种是基于后对齐的双塔架构，另一种是将文本上下文、音频和视频混合在共享空间中的全统一三模态设计。前者削弱了细粒度的音视频协同演化，后者则将语义条件与底层同步耦合。为克服这些局限，我们提出NAVA——一种面向联合音视频生成的原生视听对齐框架。NAVA基于上下文条件化的原生视听对齐：首先在专用交互空间中建立音视频对应关系，再利用外部上下文约束联合去噪过程。具体而言，NAVA采用"先对齐后融合"的MMDiT架构实现，该架构从模态感知的音视频对齐过渡到模态共享的联合去噪。此外，我们引入上下文音色条件机制，将参考音色线索与对应语音片段关联，以实现可控的语音音色。在Verse-Bench和Seed-TTS上的实验及用户研究表明，NAVA仅用6.3B参数即可实现卓越的视频质量、精确的视听同步、具有竞争力的音频质量，以及更强的参考音色可控性。

English

Joint audio-video generation aims to synthesize temporally synchronized and semantically coherent visual-acoustic content. However, existing open-source methods mainly rely on either dual-tower designs with posterior alignment or fully unified tri-modal designs that mix textual context, audio and video in one shared space. The former weakens fine-grained audio-video co-evolution, while the latter couples semantic conditioning with low-level synchronization. To address these limitations, we propose NAVA, a Native Audio-Visual Alignment framework for joint audio-video generation. NAVA is built upon context-conditioned native audio-visual alignment: it first establishes audio-video correspondence in a dedicated interaction space, and then uses external context to condition the joint denoising process. Specifically, NAVA is instantiated with an Align-then-Fuse MMDiT architecture, which transitions from modality-aware audio-video alignment to modality-shared joint denoising. Furthermore, we introduce Timbre-in-Context Conditioning to associate reference timbre cues with corresponding speech spans to achieve controllable speech timbre. Experiments on Verse-Bench and Seed-TTS, together with a user study, demonstrate that NAVA achieves superior video quality, precise audio-visual synchronization, competitive audio quality, and stronger reference-timbre controllability using only 6.3B parameters.