當今的大型語言模型能否妥善闡釋幸福概念？

摘要

福祉涵蓋了對個人成長及明智生活決策至關重要的心理、生理與社會層面。隨著越來越多的人諮詢大型語言模型（LLMs）以理解福祉，一個關鍵挑戰浮現：LLMs能否生成不僅準確且能適應多元受眾的解釋？高品質的解釋既需事實正確性，也需滿足不同專業背景用戶的期望。本研究構建了一個大規模數據集，包含由十種多樣化LLMs生成的2,194個福祉概念的43,880條解釋。我們引入了一種基於原則的LLM作為評判者的評估框架，採用雙重評判者來評估解釋質量。此外，我們展示了通過監督微調（SFT）和直接偏好優化（DPO）對開源LLM進行微調，能顯著提升生成解釋的質量。研究結果揭示：（1）所提出的LLM評判者與人類評估高度一致；（2）解釋質量在模型、受眾及類別間存在顯著差異；（3）經DPO和SFT微調的模型超越其更大規模的對照組，證明了基於偏好的學習在專門解釋任務中的有效性。

English

Well-being encompasses mental, physical, and social dimensions essential to personal growth and informed life decisions. As individuals increasingly consult Large Language Models (LLMs) to understand well-being, a key challenge emerges: Can LLMs generate explanations that are not only accurate but also tailored to diverse audiences? High-quality explanations require both factual correctness and the ability to meet the expectations of users with varying expertise. In this work, we construct a large-scale dataset comprising 43,880 explanations of 2,194 well-being concepts, generated by ten diverse LLMs. We introduce a principle-guided LLM-as-a-judge evaluation framework, employing dual judges to assess explanation quality. Furthermore, we show that fine-tuning an open-source LLM using Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) can significantly enhance the quality of generated explanations. Our results reveal: (1) The proposed LLM judges align well with human evaluations; (2) explanation quality varies significantly across models, audiences, and categories; and (3) DPO- and SFT-finetuned models outperform their larger counterparts, demonstrating the effectiveness of preference-based learning for specialized explanation tasks.

當今的大型語言模型能否妥善闡釋幸福概念？

Are Today's LLMs Ready to Explain Well-Being Concepts?

摘要

Support