VBench-2.0：推进视频生成基准套件，提升内在真实性评估

VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness

March 27, 2025

作者: Dian Zheng, Ziqi Huang, Hongbo Liu, Kai Zou, Yinan He, Fan Zhang, Yuanhan Zhang, Jingwen He, Wei-Shi Zheng, Yu Qiao, Ziwei Liu

cs.AI

摘要

视频生成技术已取得显著进展，从生成不真实的输出，发展到能够制作出视觉上令人信服且时间连贯的视频。为了评估这些视频生成模型，诸如VBench等基准测试被开发出来，用以衡量其忠实度，包括单帧美学、时间一致性及基本提示遵循等因素。然而，这些方面主要代表了表面上的忠实度，即关注视频是否在视觉上令人信服，而非是否遵循现实世界原则。尽管近期模型在这些指标上表现越来越好，但在生成不仅视觉上合理而且本质上真实的视频方面仍面临挑战。为了通过视频生成实现真正的“世界模型”，下一个前沿在于内在忠实度，确保生成的视频遵循物理定律、常识推理、解剖学正确性及构图完整性。达到这一层次的真实感对于AI辅助电影制作和模拟世界建模等应用至关重要。为弥合这一差距，我们推出了VBench-2.0，这是一款旨在自动评估视频生成模型内在忠实度的新一代基准测试。VBench-2.0评估五个关键维度：人类逼真度、可控性、创造力、物理性和常识性，每个维度进一步细分为精细能力。我们的评估框架针对各维度量身定制，整合了如最先进的视觉语言模型（VLMs）和大型语言模型（LLMs）等通用工具，以及专为视频生成提出的异常检测方法等专业工具。我们进行了广泛的标注工作，以确保与人类判断的一致性。通过超越表面忠实度，追求内在忠实度，VBench-2.0旨在为下一代视频生成模型设定新的内在忠实度标准。

English

Video generation has advanced significantly, evolving from producing unrealistic outputs to generating videos that appear visually convincing and temporally coherent. To evaluate these video generative models, benchmarks such as VBench have been developed to assess their faithfulness, measuring factors like per-frame aesthetics, temporal consistency, and basic prompt adherence. However, these aspects mainly represent superficial faithfulness, which focus on whether the video appears visually convincing rather than whether it adheres to real-world principles. While recent models perform increasingly well on these metrics, they still struggle to generate videos that are not just visually plausible but fundamentally realistic. To achieve real "world models" through video generation, the next frontier lies in intrinsic faithfulness to ensure that generated videos adhere to physical laws, commonsense reasoning, anatomical correctness, and compositional integrity. Achieving this level of realism is essential for applications such as AI-assisted filmmaking and simulated world modeling. To bridge this gap, we introduce VBench-2.0, a next-generation benchmark designed to automatically evaluate video generative models for their intrinsic faithfulness. VBench-2.0 assesses five key dimensions: Human Fidelity, Controllability, Creativity, Physics, and Commonsense, each further broken down into fine-grained capabilities. Tailored for individual dimensions, our evaluation framework integrates generalists such as state-of-the-art VLMs and LLMs, and specialists, including anomaly detection methods proposed for video generation. We conduct extensive annotations to ensure alignment with human judgment. By pushing beyond superficial faithfulness toward intrinsic faithfulness, VBench-2.0 aims to set a new standard for the next generation of video generative models in pursuit of intrinsic faithfulness.