Sora以令人驚嘆的幾何一致性生成視頻。

Sora Generates Videos with Stunning Geometrical Consistency

February 27, 2024

作者: Xuanyi Li, Daquan Zhou, Chenxu Zhang, Shaodong Wei, Qibin Hou, Ming-Ming Cheng

cs.AI

摘要

最近開發的 Sora 模型[1] 在影片生成方面展現出卓越的能力，引發了對其模擬真實世界現象能力的激烈討論。儘管其日益普及，卻缺乏建立的指標來定量評估其與真實世界物理的符合度。本文引入了一個新的基準，評估生成影片的質量，基於其遵循真實世界物理原則的程度。我們採用一種方法將生成的影片轉換為3D模型，利用這樣的前提：3D重建的準確性在很大程度上取決於影片質量。從3D重建的角度來看，我們使用構建的3D模型滿足的幾何約束的忠實度作為一個代理，來衡量生成的影片符合真實世界物理規則的程度。專案頁面：https://sora-geometrical-consistency.github.io/

English

The recently developed Sora model [1] has exhibited remarkable capabilities in video generation, sparking intense discussions regarding its ability to simulate real-world phenomena. Despite its growing popularity, there is a lack of established metrics to evaluate its fidelity to real-world physics quantitatively. In this paper, we introduce a new benchmark that assesses the quality of the generated videos based on their adherence to real-world physics principles. We employ a method that transforms the generated videos into 3D models, leveraging the premise that the accuracy of 3D reconstruction is heavily contingent on the video quality. From the perspective of 3D reconstruction, we use the fidelity of the geometric constraints satisfied by the constructed 3D models as a proxy to gauge the extent to which the generated videos conform to real-world physics rules. Project page: https://sora-geometrical-consistency.github.io/