Steer2Adapt：动态组合导向向量实现大型语言模型的高效自适应

摘要

激活导向技术已成为高效调整大语言模型以适应下游行为的有效方法。然而现有导向方法大多依赖每个任务或概念的单一静态方向，导致其难以应对任务变化，也无法胜任需要多维度协调能力的复杂任务。为此，我们提出STEER2ADAPT轻量级框架，通过组合导向向量而非从头学习新向量来实现大语言模型适配。在推理、安全等诸多领域，不同任务往往共享少量底层概念维度。STEER2ADAPT将这些维度捕获为可复用的低维语义先验子空间，仅需少量示例即可通过动态发现基向量的线性组合来适应新任务。在推理与安全领域的9项任务和3种模型上的实验表明，STEER2ADAPT平均性能提升达8.2%。深入分析进一步揭示，该框架是一种数据高效、稳定性强且透明度高的大语言模型推理时适配方法。

English

Activation steering has emerged as a promising approach for efficiently adapting large language models (LLMs) to downstream behaviors. However, most existing steering methods rely on a single static direction per task or concept, making them inflexible under task variation and inadequate for complex tasks that require multiple coordinated capabilities. To address this limitation, we propose STEER2ADAPT, a lightweight framework that adapts LLMs by composing steering vectors rather than learning new ones from scratch. In many domains (e.g., reasoning or safety), tasks share a small set of underlying concept dimensions. STEER2ADAPT captures these dimensions as a reusable, low-dimensional semantic prior subspace, and adapts to new tasks by dynamically discovering a linear combination of basis vectors from only a handful of examples. Experiments across 9 tasks and 3 models in both reasoning and safety domains demonstrate the effectiveness of STEER2ADAPT, achieving an average improvement of 8.2%. Extensive analyses further show that STEER2ADAPT is a data-efficient, stable, and transparent inference-time adaptation method for LLMs.

Steer2Adapt：动态组合导向向量实现大型语言模型的高效自适应

Steer2Adapt: Dynamically Composing Steering Vectors Elicits Efficient Adaptation of LLMs

摘要

Support