VINO：一种具有交错式全模态上下文的统一视觉生成器

摘要

我们推出VINO——一个在单一框架内实现图像与视频生成及编辑的统一视觉生成器。不同于依赖任务专用模型或独立模态模块的传统方案，VINO采用共享的扩散主干网络，能够同时接受文本、图像和视频作为条件输入，从而在单一模型中实现广泛的视觉创作与编辑任务。具体而言，VINO将视觉语言模型（VLM）与多模态扩散Transformer（MMDiT）相结合，将多模态输入编码为交错的条件标记，进而引导扩散过程。该设计支持多参考锚定、长指令跟随以及静态动态内容间的连贯身份保持，同时避免了模态专用的架构组件。为训练这一统一系统，我们提出了多阶段训练流程，逐步将基础视频生成模型扩展为能同时处理图像与视频输入输出的统一多任务生成器。在多样化生成与编辑基准测试中，VINO展现出卓越的视觉质量、精准的指令跟随能力、优化的参考与属性保持效果，以及更可控的多身份编辑性能。我们的研究成果揭示了可扩展统一视觉生成的实际路径，并验证了交错式上下文计算作为通用视觉创作基础框架的巨大潜力。

English

We present VINO, a unified visual generator that performs image and video generation and editing within a single framework. Instead of relying on task-specific models or independent modules for each modality, VINO uses a shared diffusion backbone that conditions on text, images and videos, enabling a broad range of visual creation and editing tasks under one model. Specifically, VINO couples a vision-language model (VLM) with a Multimodal Diffusion Transformer (MMDiT), where multimodal inputs are encoded as interleaved conditioning tokens, and then used to guide the diffusion process. This design supports multi-reference grounding, long-form instruction following, and coherent identity preservation across static and dynamic content, while avoiding modality-specific architectural components. To train such a unified system, we introduce a multi-stage training pipeline that progressively expands a video generation base model into a unified, multi-task generator capable of both image and video input and output. Across diverse generation and editing benchmarks, VINO demonstrates strong visual quality, faithful instruction following, improved reference and attribute preservation, and more controllable multi-identity edits. Our results highlight a practical path toward scalable unified visual generation, and the promise of interleaved, in-context computation as a foundation for general-purpose visual creation.

VINO：一种具有交错式全模态上下文的统一视觉生成器

VINO: A Unified Visual Generator with Interleaved OmniModal Context

摘要

Support