ChatPaper.aiChatPaper

GGSS:用于生成式视觉-语言模型推理时去偏的测地线门控球面引导方法

GGSS: Geodesic-Gated Spherical Steering for Inference-Time Debiasing of Generative Vision-Language Models

August 26, 2026
作者: Yiqun Sun, Junyu Chen, Pengfei Wei, Lawrence B. Hsieh
cs.AI

摘要

生成式视觉语言模型(VLMs)越来越多地应用于以人为中心的环境,但即使在图像仅在感知种族或性别等受控属性上存在差异时,它们仍可能产生人口统计偏差输出。然而,现有的推理时去偏器主要针对静态嵌入或类CLIP模型设计,而非生成式视觉语言模型。我们提出GGSS——测地线门控球面引导——一种保范干预方法,在单位超球面上发现反事实偏差子空间,沿测地线弧引导视觉标记,并使用自适应门控将修正集中于携带更强人口统计信号的标记上。我们在统一的单工作点协议下,针对十个适配的推理时去偏基线方法和基于提示词的缓解方法,对四种生成式视觉语言模型进行评测,涵盖类别、成对和职业-性别偏差测试,同时评估其通用视觉语言能力。GGSS在所有四个模型上均取得最低的平均偏差,在四个骨干网络中的三个上通过配对置换检验达到显著水平,同时将MMStar准确率保持在未引导基线±0.6个百分点以内。代码可在 https://github.com/dukesun99/GGSS 获取。
English
Generative vision-language models (VLMs) are increasingly used in human-centered settings, yet they can produce demographically biased outputs even when images differ only in controlled attributes such as perceived race or gender. However, existing inference-time debiasers were largely designed for static embeddings or CLIP-like models rather than generative VLMs. We propose GGSS---Geodesic-Gated Spherical Steering---a norm-preserving intervention that discovers a counterfactual bias subspace on the unit hypersphere, steers visual tokens along geodesic arcs, and uses an adaptive gate to focus correction on tokens that carry stronger demographic signal. We evaluate four generative VLMs against ten adapted inference-time debiasing baselines and prompt-based mitigation under a single operating-point protocol across categorical, pairwise, and occupation-gender bias tests, while also measuring general visual-language capability. GGSS achieves the lowest average bias on all four models, significant on three of four backbones under paired permutation tests, while preserving MMStar accuracy within +/- 0.6 p.p. of the unsteered baseline. Code is available at https://github.com/dukesun99/GGSS.