当今的大型语言模型能否胜任解释幸福概念的任务？

摘要

福祉涵盖了心理、生理和社会等多个维度，这些维度对个人成长和明智的生活决策至关重要。随着越来越多的人向大型语言模型（LLMs）咨询以理解福祉，一个关键问题浮现：LLMs能否生成不仅准确，还能适应不同受众需求的解释？高质量的解释既需要事实的正确性，也要能够满足不同专业背景用户的期望。在本研究中，我们构建了一个大规模数据集，包含由十个多样化LLMs生成的2,194个福祉概念的43,880条解释。我们引入了一种基于原则的LLM作为评判者的评估框架，采用双重评判者机制来评估解释质量。此外，我们展示了通过监督微调（SFT）和直接偏好优化（DPO）对开源LLM进行微调，可以显著提升生成解释的质量。我们的研究结果表明：（1）所提出的LLM评判者与人类评估高度一致；（2）解释质量在模型、受众和类别之间存在显著差异；（3）经过DPO和SFT微调的模型在性能上超越了规模更大的模型，证明了基于偏好的学习在专门解释任务中的有效性。

English

Well-being encompasses mental, physical, and social dimensions essential to personal growth and informed life decisions. As individuals increasingly consult Large Language Models (LLMs) to understand well-being, a key challenge emerges: Can LLMs generate explanations that are not only accurate but also tailored to diverse audiences? High-quality explanations require both factual correctness and the ability to meet the expectations of users with varying expertise. In this work, we construct a large-scale dataset comprising 43,880 explanations of 2,194 well-being concepts, generated by ten diverse LLMs. We introduce a principle-guided LLM-as-a-judge evaluation framework, employing dual judges to assess explanation quality. Furthermore, we show that fine-tuning an open-source LLM using Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) can significantly enhance the quality of generated explanations. Our results reveal: (1) The proposed LLM judges align well with human evaluations; (2) explanation quality varies significantly across models, audiences, and categories; and (3) DPO- and SFT-finetuned models outperform their larger counterparts, demonstrating the effectiveness of preference-based learning for specialized explanation tasks.

当今的大型语言模型能否胜任解释幸福概念的任务？

Are Today's LLMs Ready to Explain Well-Being Concepts?

摘要

Support