T2AV-Compass：迈向文本到音视频生成的统一评估框架

摘要

文本-音视频生成技术旨在从自然语言中合成时序连贯的视频与语义同步的音频，然而其评估体系仍处于碎片化状态，常依赖单模态指标或范围狭窄的基准测试，难以捕捉复杂提示下的跨模态对齐、指令遵循及感知真实度。为突破此局限，我们提出T2AV-Compass——一个用于全面评估T2AV系统的统一基准，包含通过分类学驱动流程构建的500个多样化复杂提示，确保语义丰富性与物理合理性。此外，T2AV-Compass引入双层级评估框架，既整合了针对视频质量、音频质量及跨模态对齐的客观信号级指标，又结合了基于MLLM即评判器的主观协议来评估指令遵循度与真实感。对11个代表性T2AV系统的广泛测试表明，即使最强模型在人类级真实感与跨模态一致性方面仍存在显著差距，且在音频真实感、细粒度同步、指令遵循等方面存在持续缺陷。这些结果揭示了未来模型的巨大改进空间，同时彰显了T2AV-Compass作为推动文本-音视频生成技术发展的挑战性诊断测试平台的价值。

English

Text-to-Audio-Video (T2AV) generation aims to synthesize temporally coherent video and semantically synchronized audio from natural language, yet its evaluation remains fragmented, often relying on unimodal metrics or narrowly scoped benchmarks that fail to capture cross-modal alignment, instruction following, and perceptual realism under complex prompts. To address this limitation, we present T2AV-Compass, a unified benchmark for comprehensive evaluation of T2AV systems, consisting of 500 diverse and complex prompts constructed via a taxonomy-driven pipeline to ensure semantic richness and physical plausibility. Besides, T2AV-Compass introduces a dual-level evaluation framework that integrates objective signal-level metrics for video quality, audio quality, and cross-modal alignment with a subjective MLLM-as-a-Judge protocol for instruction following and realism assessment. Extensive evaluation of 11 representative T2AVsystems reveals that even the strongest models fall substantially short of human-level realism and cross-modal consistency, with persistent failures in audio realism, fine-grained synchronization, instruction following, etc. These results indicate significant improvement room for future models and highlight the value of T2AV-Compass as a challenging and diagnostic testbed for advancing text-to-audio-video generation.

T2AV-Compass：迈向文本到音视频生成的统一评估框架

T2AV-Compass: Towards Unified Evaluation for Text-to-Audio-Video Generation

摘要

Support