LLM-DetectAIve：一個用於細粒度機器生成文本檢測的工具

LLM-DetectAIve: a Tool for Fine-Grained Machine-Generated Text Detection

August 8, 2024

作者: Mervat Abassy, Kareem Elozeiri, Alexander Aziz, Minh Ngoc Ta, Raj Vardhan Tomar, Bimarsha Adhikari, Saad El Dine Ahmed, Yuxia Wang, Osama Mohammed Afzal, Zhuohan Xie, Jonibek Mansurov, Ekaterina Artemova, Vladislav Mikhailov, Rui Xing, Jiahui Geng, Hasan Iqbal, Zain Muhammad Mujahid, Tarek Mahmoud, Akim Tsvigun, Alham Fikri Aji, Artem Shelmanov, Nizar Habash, Iryna Gurevych, Preslav Nakov

cs.AI

摘要

大型語言模型（LLMs）廣泛地對一般大眾開放，顯著擴大了機器生成文本（MGTs）的傳播。提示操作的進展加劇了識別文本來源（人類撰寫 vs 機器生成）的困難。這引發了對MGTs潛在濫用的擔憂，特別是在教育和學術領域。本文介紹了LLM-DetectAIve，這是一個用於細粒度MGT檢測的系統。它能夠將文本分類為四個類別：人類撰寫、機器生成、機器撰寫機器人化、以及人類撰寫機器精緻。與以往執行二元分類的MGT檢測器不同，LLM-DetectAIve引入了兩個額外的類別，提供了有關LLM在文本創建過程中干預程度不同的見解。這在一些領域可能很有用，比如教育領域，通常禁止任何LLM干預。實驗表明，LLM-DetectAIve能夠有效識別文本內容的作者，證明了它在增強教育、學術和其他領域的誠信方面的用途。LLM-DetectAIve可在https://huggingface.co/spaces/raj-tomar001/MGT-New 公開訪問。介紹我們系統的視頻可在https://youtu.be/E8eT_bE7k8c觀看。

English

The widespread accessibility of large language models (LLMs) to the general public has significantly amplified the dissemination of machine-generated texts (MGTs). Advancements in prompt manipulation have exacerbated the difficulty in discerning the origin of a text (human-authored vs machinegenerated). This raises concerns regarding the potential misuse of MGTs, particularly within educational and academic domains. In this paper, we present LLM-DetectAIve -- a system designed for fine-grained MGT detection. It is able to classify texts into four categories: human-written, machine-generated, machine-written machine-humanized, and human-written machine-polished. Contrary to previous MGT detectors that perform binary classification, introducing two additional categories in LLM-DetectiAIve offers insights into the varying degrees of LLM intervention during the text creation. This might be useful in some domains like education, where any LLM intervention is usually prohibited. Experiments show that LLM-DetectAIve can effectively identify the authorship of textual content, proving its usefulness in enhancing integrity in education, academia, and other domains. LLM-DetectAIve is publicly accessible at https://huggingface.co/spaces/raj-tomar001/MGT-New. The video describing our system is available at https://youtu.be/E8eT_bE7k8c.