音乐竞技场：文本到音乐的实时评估
Music Arena: Live Evaluation for Text-to-Music
July 28, 2025
作者: Yonghyun Kim, Wayne Chi, Anastasios N. Angelopoulos, Wei-Lin Chiang, Koichi Saito, Shinji Watanabe, Yuki Mitsufuji, Chris Donahue
cs.AI
摘要
我们推出Music Arena，一个用于文本到音乐（TTM）模型可扩展人类偏好评估的开放平台。通过听力研究征求人类偏好是TTM评估的黄金标准，但这些研究成本高昂且难以比较，因为不同系统的研究协议可能各异。此外，人类偏好或许能帮助研究人员调整其TTM系统或改进自动评估指标，但目前尚不存在一个开放且可更新的偏好来源。我们旨在通过提供TTM的*实时*评估来填补这些空白。在Music Arena中，真实用户输入自选的文本提示，并比较两个TTM系统的输出，他们的偏好被用来编制排行榜。尽管Music Arena遵循了其他AI领域近期的评估趋势，我们也为其设计了针对音乐的关键特性：一个基于LLM的路由系统，以导航TTM系统的异质类型签名，以及收集*详细*偏好，包括听力数据和自然语言反馈。我们还提出了一项滚动数据发布政策，确保用户隐私，提供可更新的偏好数据源，并增加平台透明度。通过其标准化的评估协议、透明的数据访问政策以及针对音乐的特性，Music Arena不仅解决了TTM生态系统中的关键挑战，还展示了如何深思熟虑地将实时评估适应于特定AI领域的独特特征。
Music Arena可通过以下网址访问：https://music-arena.org
English
We present Music Arena, an open platform for scalable human preference
evaluation of text-to-music (TTM) models. Soliciting human preferences via
listening studies is the gold standard for evaluation in TTM, but these studies
are expensive to conduct and difficult to compare, as study protocols may
differ across systems. Moreover, human preferences might help researchers align
their TTM systems or improve automatic evaluation metrics, but an open and
renewable source of preferences does not currently exist. We aim to fill these
gaps by offering *live* evaluation for TTM. In Music Arena, real-world users
input text prompts of their choosing and compare outputs from two TTM systems,
and their preferences are used to compile a leaderboard. While Music Arena
follows recent evaluation trends in other AI domains, we also design it with
key features tailored to music: an LLM-based routing system to navigate the
heterogeneous type signatures of TTM systems, and the collection of *detailed*
preferences including listening data and natural language feedback. We also
propose a rolling data release policy with user privacy guarantees, providing a
renewable source of preference data and increasing platform transparency.
Through its standardized evaluation protocol, transparent data access policies,
and music-specific features, Music Arena not only addresses key challenges in
the TTM ecosystem but also demonstrates how live evaluation can be thoughtfully
adapted to unique characteristics of specific AI domains.
Music Arena is available at: https://music-arena.org