利用人類偏好獎勵改善文本到音樂生成

摘要

我們描述了我們在ICME 2026學術文本轉音樂（ATTM）Grand Challenge效率競賽中的參賽方法。除了挑戰協議中的FAD-CLAP和CLAP分數之外，我們還加入了來自TuneJury的學習型人類偏好獎勵，TuneJury是一個基於開放音樂偏好數據集訓練的雙向成對排序器。該獎勵既作為訓練時的條件信號，也作為樣本選擇的標準。該流程結合了五項工程決策，基於一個1.2億參數的FluxAudio-S主幹網路，其中四項在訓練時執行，一項在推理時執行：(i) 訓練時的獎勵條件化，同時作為推理時的CFG軸；(ii) 對五種分數條件化架構進行掃描，訓練和推理使用不同的變體；(iii) 對前十分位樣本進行專家迭代；(iv) 使用短偏好調優（CRPO）進行音頻-文本對齊；(v) 通過聯合CFG、源分離和響度歸一化進行推理後處理。基於100個Song Describer提示的逐階段分解顯示，訓練時的獎勵條件化是一個功能性的條件化軸，專家迭代是主要貢獻因素，偏好調優僅帶來噪聲級別的增益，而推理時的分數標量在流程結束時已經飽和。

English

We describe our entry to the efficiency track of the Academic Text-to-Music (ATTM) Grand Challenge at ICME 2026. Beyond the challenge protocol's FAD-CLAP and CLAP score, we add a learned human-preference reward from TuneJury, a twin pairwise ranker trained over open music-preference datasets. The reward serves both as a training-time conditioning signal and as a sample-selection criterion. The pipeline combines five engineering decisions on a 120M-parameter FluxAudio-S backbone, four at training time and one at inference: (i) training-time reward conditioning that doubles as an inference-time CFG axis, (ii) a sweep over five score-conditioning architectures, where training and inference use different variants, (iii) expert iteration on the top decile, (iv) a short preference-tuning pass (CRPO) for audio-text alignment, and (v) inference post-processing via joint CFG, source separation, and loudness normalization. Per-stage decomposition on 100 Song Describer prompts shows training-time reward conditioning as a functional conditioning axis, expert iteration as the dominant contributor, the preference-tuning pass adding only noise-level gain, and the inference-time score scalar already saturated by the end of the chain.