對齊生成式音樂AI與人類偏好：方法與挑戰

摘要

近期音樂生成式AI雖在保真度與風格多樣性方面取得顯著進展，但由於所採用的特定損失函數，這些系統往往難以契合人類細膩的偏好。本文主張將偏好對齊技術系統化應用於音樂生成領域，以彌合計算優化與人類音樂審美之間的根本差距。借鑒包括MusicRL大規模偏好學習、DiffRhythm+中基於擴散模型的偏好優化等多偏好對齊框架，以及Text2midi-InferAlign等推論時優化技術在內的最新突破，我們探討這些技術如何應對音樂特有的挑戰：時序連貫性、和聲一致性與主觀質量評估。我們指出關鍵研究挑戰包括長篇作曲的可擴展性、偏好建模的可靠性等。展望未來，我們預見偏好對齊的音樂生成將為互動式作曲工具與個性化音樂服務帶來變革性應用。本研究呼籲持續開展跨領域合作，結合機器學習與音樂理論的進展，創建真正服務於人類創作與體驗需求的音樂AI系統。

English

Recent advances in generative AI for music have achieved remarkable fidelity and stylistic diversity, yet these systems often fail to align with nuanced human preferences due to the specific loss functions they use. This paper advocates for the systematic application of preference alignment techniques to music generation, addressing the fundamental gap between computational optimization and human musical appreciation. Drawing on recent breakthroughs including MusicRL's large-scale preference learning, multi-preference alignment frameworks like diffusion-based preference optimization in DiffRhythm+, and inference-time optimization techniques like Text2midi-InferAlign, we discuss how these techniques can address music's unique challenges: temporal coherence, harmonic consistency, and subjective quality assessment. We identify key research challenges including scalability to long-form compositions, reliability amongst others in preference modelling. Looking forward, we envision preference-aligned music generation enabling transformative applications in interactive composition tools and personalized music services. This work calls for sustained interdisciplinary research combining advances in machine learning, music-theory to create music AI systems that truly serve human creative and experiential needs.

對齊生成式音樂AI與人類偏好：方法與挑戰

Aligning Generative Music AI with Human Preferences: Methods and Challenges

摘要

Support