何为伤害？通过人类中心研究量化机器翻译中性别偏见的实际影响

摘要

机器翻译（MT）中的性别偏见被认为是一个可能伤害人们和社会的问题。然而，该领域的进展很少涉及最终的MT用户，也很少告知他们可能受到偏见技术影响的方式。目前的评估通常局限于自动方法，这些方法提供了性别差异可能带来的下游影响的不透明估计。我们进行了一项广泛的以人为中心的研究，以检查MT中的偏见是否带来了实质性成本，如服务质量差距在女性和男性之间。为此，我们从90名参与者那里收集了行为数据，他们对MT输出进行了后期编辑，以确保正确的性别翻译。在多个数据集、语言和用户类型中，我们的研究表明，女性后期编辑明显需要更多的技术和时间投入，也对应着更高的财务成本。然而，现有的偏见测量未能反映出发现的差异。我们的发现倡导以人为中心的方法，可以告知偏见的社会影响。

English

Gender bias in machine translation (MT) is recognized as an issue that can harm people and society. And yet, advancements in the field rarely involve people, the final MT users, or inform how they might be impacted by biased technologies. Current evaluations are often restricted to automatic methods, which offer an opaque estimate of what the downstream impact of gender disparities might be. We conduct an extensive human-centered study to examine if and to what extent bias in MT brings harms with tangible costs, such as quality of service gaps across women and men. To this aim, we collect behavioral data from 90 participants, who post-edited MT outputs to ensure correct gender translation. Across multiple datasets, languages, and types of users, our study shows that feminine post-editing demands significantly more technical and temporal effort, also corresponding to higher financial costs. Existing bias measurements, however, fail to reflect the found disparities. Our findings advocate for human-centered approaches that can inform the societal impact of bias.

何为伤害？通过人类中心研究量化机器翻译中性别偏见的实际影响

What the Harm? Quantifying the Tangible Impact of Gender Bias in Machine Translation with a Human-centered Study

摘要

Support