公平性剪枝：通过差异激活定位GLU-MLP层中的群体偏差

摘要

本文提出了公平剪枝（Fairness Pruning），一种轻量级结构性干预方法，旨在用于大语言模型（LLMs）中人口统计偏差的管理与未来缓解。作为对该方法的初步实证验证，本文聚焦于因果偏差定位。通过使用最小对比提示对和推理时激活捕获，该方法能够识别GLU架构中在加工人口统计属性时产生差异性反应的神经元，并在down_proj输入处评估信号。实证评估在高达30亿参数的模型（Llama-3.2系列与Salamandra-2B）上进行，结合了标准化基准评估与定性文本生成实验。结果表明，将识别出的神经元置零会改变模型对相关人口统计变量的响应方式。然而，干预并未产生平坦化的缓解效果，而是引发了双向偏差失稳：由于BiasScore是无符号的，候选集混合了分别推动刻板印象增强与减弱方向的神经元，而聚合偏差的净效应取决于何种符号占主导。该干预极具外科手术式精度：在Llama-3.2-1B中至多置零40个神经元（不到总MLP宽度的0.031%），即可在推理和通用知识能力上实现99.49%的平均保持率。这些发现从经验上证实，人口统计偏差加工与模型能力运行于可分离的回路之上，从而为从盲目置零转向定向行为调制奠定了方法论基础。

English

This work presents Fairness Pruning, a lightweight structural intervention method designed for the management and future mitigation of demographic bias in large language models (LLMs). As a foundational empirical validation of this method, this work focuses on causal bias localization. Using minimally contrastive prompt pairs and inference-time activation capture, the method identifies neurons that react differentially when processing demographic attributes in GLU architectures, evaluating the signal at the down_proj input. Empirical evaluation was conducted on models of up to 3 billion parameters (Llama-3.2 family and Salamandra-2B), combining standardized benchmark evaluation with qualitative text generation experiments. Results demonstrate that zeroing the identified neurons alters how the model responds to associated demographic variables. However, rather than producing flat mitigation, the intervention causes bidirectional bias destabilization: because BiasScore is unsigned, candidate sets mix neurons that push toward and against the stereotype, and the net effect on aggregate bias depends on which sign dominates. The intervention is extremely surgical: zeroing at most 40 neurons in Llama-3.2-1B (less than 0.031% of total MLP width) achieves a mean retention of 99.49% in reasoning and general knowledge capabilities. These findings empirically confirm that demographic bias processing and model capabilities operate on dissociable circuits, establishing the methodological foundations for transitioning from blind zeroing toward directional behavior modulation.