面向低资源语言的Tatoxa文本去毒化系统：以鞑靼语为例

摘要

文本解毒（即自动检测和减轻辱骂性与有害内容）对于保障在线社区安全及保护用户至关重要。然而，诸如鞑靼语这类低资源语言的研究却鲜有关注。本文提出Tatoxa——一种针对鞑靼语文本解毒的最新先进系统。对比实验表明，所提方法在关键质量指标上优于现有开源及专有的商业大型语言模型。我们还引入了一个专为鞑靼语文本解毒设计的新数据集，适用于低资源场景下的微调与评估。最后，跨语言迁移实验显示，即使存在大规模的俄语语料库，从其他语言（包括文化相近的俄语）进行迁移的效果也显著逊色于基于本地鞑靼语数据的训练。

English

Text detoxification, the automated detection and mitigation of abusive and harmful content, is essential for ensuring the safety of online communities and protecting users. However, low resource languages such as Tatar have received little research attention. In this paper we present Tatoxa, a novel state-of-the-art system for text detoxification in the Tatar language. Comparative experiments show that the proposed approach outperforms existing open source and proprietary commercial LLMs on key quality metrics. We also introduce a new dataset for text detoxification in Tatar, designed for fine tuning and evaluation in low resource settings. Finally, cross lingual transfer experiments indicate that transfer from other languages, including the culturally close Russian, performs significantly worse than training on native Tatar data even when a large Russian corpus is available.