通过“定义问题测试”探索大型语言模型的认知道德发展。

Exploring Large Language Models' Cognitive Moral Development through Defining Issues Test

September 23, 2023

作者: Kumar Tanmay, Aditi Khandelwal, Utkarsh Agarwal, Monojit Choudhury

cs.AI

摘要

大型语言模型的发展引起了研究人员广泛的兴趣，他们希望了解这些模型固有的推理和问题解决能力。尽管有大量研究致力于阐明这些能力，但对于这些模型的道德发展和判断仍存在相当大的认知差距。目前评估这些模型道德推理能力的方法是将其视为分类任务，但由于过度简化，存在许多不准确之处。在本研究中，我们通过搭建人类心理学和人工智能这两个不同领域之间的联系，提出了一个有效的评估框架，可以借助心理测量评估工具-道德问题定义测验，帮助描绘模型的道德推理能力，包括道德一致性和科尔伯格的道德发展阶段。

English

The development of large language models has instilled widespread interest among the researchers to understand their inherent reasoning and problem-solving capabilities. Despite good amount of research going on to elucidate these capabilities, there is a still an appreciable gap in understanding moral development and judgments of these models. The current approaches of evaluating the ethical reasoning abilities of these models as a classification task pose numerous inaccuracies because of over-simplification. In this study, we built a psychological connection by bridging two disparate fields-human psychology and AI. We proposed an effective evaluation framework which can help to delineate the model's ethical reasoning ability in terms of moral consistency and Kohlberg's moral development stages with the help of Psychometric Assessment Tool-Defining Issues Test.