WildScore：多模态大语言模型在真实场景下的符号音乐推理基准测试

摘要

近期，多模态大语言模型（MLLMs）在多种视觉-语言任务中展现了令人瞩目的能力。然而，其在多模态符号音乐领域的推理能力仍鲜有探索。我们推出了WildScore，这是首个面向真实场景的多模态符号音乐推理与分析基准，旨在评估MLLMs解读现实世界乐谱及回答复杂音乐学问题的能力。WildScore中的每个实例均源自真实的音乐作品，并附有用户生成的真实问题与讨论，捕捉了实际音乐分析的细微之处。为促进系统化评估，我们提出了一套系统分类法，包含高层次与细粒度的音乐学本体。此外，我们将复杂的音乐推理问题转化为多项选择题形式，从而实现对MLLMs符号音乐理解能力的可控且可扩展的评估。在WildScore上对前沿MLLMs进行的实证基准测试揭示了其在视觉-符号推理中的有趣模式，既指明了MLLMs在符号音乐推理与分析中的潜在发展方向，也揭示了其面临的持续挑战。我们公开了数据集与代码。

English

Recent advances in Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities across various vision-language tasks. However, their reasoning abilities in the multimodal symbolic music domain remain largely unexplored. We introduce WildScore, the first in-the-wild multimodal symbolic music reasoning and analysis benchmark, designed to evaluate MLLMs' capacity to interpret real-world music scores and answer complex musicological queries. Each instance in WildScore is sourced from genuine musical compositions and accompanied by authentic user-generated questions and discussions, capturing the intricacies of practical music analysis. To facilitate systematic evaluation, we propose a systematic taxonomy, comprising both high-level and fine-grained musicological ontologies. Furthermore, we frame complex music reasoning as multiple-choice question answering, enabling controlled and scalable assessment of MLLMs' symbolic music understanding. Empirical benchmarking of state-of-the-art MLLMs on WildScore reveals intriguing patterns in their visual-symbolic reasoning, uncovering both promising directions and persistent challenges for MLLMs in symbolic music reasoning and analysis. We release the dataset and code.

WildScore：多模态大语言模型在真实场景下的符号音乐推理基准测试

WildScore: Benchmarking MLLMs in-the-Wild Symbolic Music Reasoning

摘要

Support