DreamDPO：透過直接偏好優化將文本生成與3D生成與人類偏好對齊

DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimization

February 5, 2025

作者: Zhenglin Zhou, Xiaobo Xia, Fan Ma, Hehe Fan, Yi Yang, Tat-Seng Chua

cs.AI

摘要

將文字轉換為3D生成自動化了從文字描述中創建3D內容，這在各個領域中具有轉變性的潛力。然而，現有方法常常難以將生成的內容與人類偏好對齊，限制了它們的應用範圍和靈活性。為了解決這些限制，在本文中，我們提出了DreamDPO，一個基於優化的框架，將人類偏好整合到3D生成過程中，通過直接偏好優化。在實踐中，DreamDPO首先構建成對示例，然後使用獎勵或大型多模型對它們與人類偏好的對齊進行比較，最後通過偏好驅動的損失函數優化3D表示。通過利用成對比較來反映偏好，DreamDPO減少了對精確點對點質量評估的依賴，同時通過偏好引導的優化實現了精細的可控性。實驗表明，DreamDPO取得了競爭性的結果，與現有方法相比提供了更高質量和更可控的3D內容。代碼和模型將開源。

English

Text-to-3D generation automates 3D content creation from textual descriptions, which offers transformative potential across various fields. However, existing methods often struggle to align generated content with human preferences, limiting their applicability and flexibility. To address these limitations, in this paper, we propose DreamDPO, an optimization-based framework that integrates human preferences into the 3D generation process, through direct preference optimization. Practically, DreamDPO first constructs pairwise examples, then compare their alignment with human preferences using reward or large multimodal models, and lastly optimizes the 3D representation with a preference-driven loss function. By leveraging pairwise comparison to reflect preferences, DreamDPO reduces reliance on precise pointwise quality evaluations while enabling fine-grained controllability through preference-guided optimization. Experiments demonstrate that DreamDPO achieves competitive results, and provides higher-quality and more controllable 3D content compared to existing methods. The code and models will be open-sourced.