数据与人工智能治理：推动大型语言模型中的公平、伦理与公正

摘要

本文探讨了在机器学习模型全生命周期中系统化治理、评估和量化偏差的方法，涵盖从初始开发与验证到持续生产监控及防护措施实施的各个环节。基于我们在大型语言模型（LLMs）偏差评估与测试套件（BEATS）上的基础性工作，作者揭示了LLMs中普遍存在的偏差与公平性相关缺陷，并讨论了针对LLMs中偏差、伦理、公平性和事实性的数据与AI治理框架。本文提出的数据与AI治理方法适用于实际应用场景，能够在LLMs投入生产前进行严格基准测试，支持持续实时评估，并主动监管LLM生成的内容。通过在AI开发全周期实施数据与AI治理，组织能够显著提升其生成式AI系统的安全性与责任感，有效降低歧视风险，防范潜在的声誉或品牌损害。最终，我们期望通过本文，为推动创建和部署符合社会责任与伦理准则的生成式人工智能应用贡献力量。

English

In this paper, we cover approaches to systematically govern, assess and quantify bias across the complete life cycle of machine learning models, from initial development and validation to ongoing production monitoring and guardrail implementation. Building upon our foundational work on the Bias Evaluation and Assessment Test Suite (BEATS) for Large Language Models, the authors share prevalent bias and fairness related gaps in Large Language Models (LLMs) and discuss data and AI governance framework to address Bias, Ethics, Fairness, and Factuality within LLMs. The data and AI governance approach discussed in this paper is suitable for practical, real-world applications, enabling rigorous benchmarking of LLMs prior to production deployment, facilitating continuous real-time evaluation, and proactively governing LLM generated responses. By implementing the data and AI governance across the life cycle of AI development, organizations can significantly enhance the safety and responsibility of their GenAI systems, effectively mitigating risks of discrimination and protecting against potential reputational or brand-related harm. Ultimately, through this article, we aim to contribute to advancement of the creation and deployment of socially responsible and ethically aligned generative artificial intelligence powered applications.

数据与人工智能治理：推动大型语言模型中的公平、伦理与公正

Data and AI governance: Promoting equity, ethics, and fairness in large language models

摘要

Support