通過「定義問題測試」探索大型語言模型的認知道德發展。

Exploring Large Language Models' Cognitive Moral Development through Defining Issues Test

September 23, 2023

作者: Kumar Tanmay, Aditi Khandelwal, Utkarsh Agarwal, Monojit Choudhury

cs.AI

摘要

大型語言模型的發展引起了研究人員廣泛的興趣，他們希望了解這些模型固有的推理和問題解決能力。儘管有大量研究致力於闡明這些能力，但對這些模型的道德發展和判斷仍存在顯著差距。目前評估這些模型道德推理能力的方法是將其視為一項分類任務，但由於過度簡化，存在許多不準確之處。在這項研究中，我們通過搭建兩個不同領域-人類心理學和人工智慧之間的心理聯繫，提出了一個有效的評估框架，可以幫助界定模型的道德推理能力，並借助心理測量評估工具-定義問題測試，來評估模型的道德一致性和柯爾伯格的道德發展階段。

English

The development of large language models has instilled widespread interest among the researchers to understand their inherent reasoning and problem-solving capabilities. Despite good amount of research going on to elucidate these capabilities, there is a still an appreciable gap in understanding moral development and judgments of these models. The current approaches of evaluating the ethical reasoning abilities of these models as a classification task pose numerous inaccuracies because of over-simplification. In this study, we built a psychological connection by bridging two disparate fields-human psychology and AI. We proposed an effective evaluation framework which can help to delineate the model's ethical reasoning ability in terms of moral consistency and Kohlberg's moral development stages with the help of Psychometric Assessment Tool-Defining Issues Test.