ToVo：透過投票的方式進行毒性分類

ToVo: Toxicity Taxonomy via Voting

June 21, 2024

作者: Tinh Son Luong, Thanh-Thien Le, Thang Viet Doan, Linh Ngo Van, Thien Huu Nguyen, Diep Thi-Ngoc Nguyen

cs.AI

摘要

現有的有毒檢測模型存在著重大限制，例如缺乏透明度、定制性和可重複性。這些挑戰源於其訓練數據的封閉性質以及對評估機制的解釋不足。為了應對這些問題，我們提出了一種數據集創建機制，該機制整合了投票和思維鏈過程，生產出一個高質量的開源數據集，用於檢測有毒內容。我們的方法確保每個樣本的多樣化分類指標，並包括分類分數以及對分類的解釋推理。 通過我們提出的機制創建的數據集來訓練我們的模型，然後將其與現有廣泛使用的檢測器進行比較。我們的方法不僅增強了透明度和定制性，還有助於更好地針對特定用例進行微調。這項工作為開發有毒內容檢測模型提供了一個堅固的框架，強調開放性和適應性，從而為更有效和用戶特定的內容審核解決方案鋪平了道路。

English

Existing toxic detection models face significant limitations, such as lack of transparency, customization, and reproducibility. These challenges stem from the closed-source nature of their training data and the paucity of explanations for their evaluation mechanism. To address these issues, we propose a dataset creation mechanism that integrates voting and chain-of-thought processes, producing a high-quality open-source dataset for toxic content detection. Our methodology ensures diverse classification metrics for each sample and includes both classification scores and explanatory reasoning for the classifications. We utilize the dataset created through our proposed mechanism to train our model, which is then compared against existing widely-used detectors. Our approach not only enhances transparency and customizability but also facilitates better fine-tuning for specific use cases. This work contributes a robust framework for developing toxic content detection models, emphasizing openness and adaptability, thus paving the way for more effective and user-specific content moderation solutions.