YINYANG-ALIGN：對立目標的基準測試並提出基於多目標優化的DPO用於文本到圖像對齊

YINYANG-ALIGN: Benchmarking Contradictory Objectives and Proposing Multi-Objective Optimization based DPO for Text-to-Image Alignment

February 5, 2025

作者: Amitava Das, Yaswanth Narsupalli, Gurpreet Singh, Vinija Jain, Vasu Sharma, Suranjana Trivedy, Aman Chadha, Amit Sheth

cs.AI

摘要

在文本到圖像（T2I）系統中，精確的對齊至關重要，以確保生成的視覺不僅準確地表達用戶意圖，還符合嚴格的道德和美學標準。像谷歌雙子星事件這樣的事件，其中不對齊的輸出引發了重大的公眾強烈反彈，突顯了強大對齊機制的關鍵性需求。相比之下，大型語言模型（LLMs）在對齊方面取得了顯著成功。借鑒這些進展，研究人員渴望將類似的對齊技術，如直接偏好優化（DPO），應用於T2I系統，以增強圖像生成的忠實度和可靠性。 我們提出了YinYangAlign，一個先進的基準框架，系統性地量化T2I系統的對齊忠實度，解決了六個基本且固有矛盾的設計目標。每一對代表圖像生成中的基本張力，例如在遵循用戶提示與進行創意修改之間取得平衡，或在視覺連貫性旁邊保持多樣性。YinYangAlign包括詳細的公理數據集，其中包括人類提示、對齊（選擇）響應、不對齊（拒絕）的AI生成輸出，以及對潛在矛盾的解釋。

English

Precise alignment in Text-to-Image (T2I) systems is crucial to ensure that generated visuals not only accurately encapsulate user intents but also conform to stringent ethical and aesthetic benchmarks. Incidents like the Google Gemini fiasco, where misaligned outputs triggered significant public backlash, underscore the critical need for robust alignment mechanisms. In contrast, Large Language Models (LLMs) have achieved notable success in alignment. Building on these advancements, researchers are eager to apply similar alignment techniques, such as Direct Preference Optimization (DPO), to T2I systems to enhance image generation fidelity and reliability. We present YinYangAlign, an advanced benchmarking framework that systematically quantifies the alignment fidelity of T2I systems, addressing six fundamental and inherently contradictory design objectives. Each pair represents fundamental tensions in image generation, such as balancing adherence to user prompts with creative modifications or maintaining diversity alongside visual coherence. YinYangAlign includes detailed axiom datasets featuring human prompts, aligned (chosen) responses, misaligned (rejected) AI-generated outputs, and explanations of the underlying contradictions.