ToVo: 通过投票实现毒性分类

ToVo: Toxicity Taxonomy via Voting

June 21, 2024

作者: Tinh Son Luong, Thanh-Thien Le, Thang Viet Doan, Linh Ngo Van, Thien Huu Nguyen, Diep Thi-Ngoc Nguyen

cs.AI

摘要

现有的毒性检测模型存在重大局限，如缺乏透明度、定制性和可复现性。这些挑战源自它们训练数据的闭源性质以及评估机制缺乏解释的问题。为了解决这些问题，我们提出了一个数据集创建机制，该机制整合了投票和思维链过程，生成了一个高质量的开源数据集，用于毒性内容检测。我们的方法确保每个样本的多样化分类指标，并包括分类分数和分类的解释推理。 我们利用通过我们提出的机制创建的数据集来训练我们的模型，然后将其与现有广泛使用的检测器进行比较。我们的方法不仅增强了透明度和定制性，还有助于更好地针对特定用例进行微调。这项工作为开发毒性内容检测模型提供了一个稳健的框架，强调开放性和适应性，从而为更有效和用户特定的内容管理解决方案铺平道路。

English

Existing toxic detection models face significant limitations, such as lack of transparency, customization, and reproducibility. These challenges stem from the closed-source nature of their training data and the paucity of explanations for their evaluation mechanism. To address these issues, we propose a dataset creation mechanism that integrates voting and chain-of-thought processes, producing a high-quality open-source dataset for toxic content detection. Our methodology ensures diverse classification metrics for each sample and includes both classification scores and explanatory reasoning for the classifications. We utilize the dataset created through our proposed mechanism to train our model, which is then compared against existing widely-used detectors. Our approach not only enhances transparency and customizability but also facilitates better fine-tuning for specific use cases. This work contributes a robust framework for developing toxic content detection models, emphasizing openness and adaptability, thus paving the way for more effective and user-specific content moderation solutions.