大型語言模型中的跨語言功能向量情緒偵測

摘要

函數向量（FVs）近來已成為一種有前景的機制，可透過注入由上下文示範萃取出的任務特定潛在方向表示，來引導大型語言模型（LLMs）的行為。雖然先前研究顯示，函數向量能在結構化上下文學習設定中恢復任務行為，但其在語意複雜任務上的有效性及跨語言泛化能力仍尚未充分探索。我們使用多語言多標籤情緒辨識作為具有挑戰性的語意分類基準，來研究函數向量的跨語言可遷移性。具體而言，我們檢驗在推論期間不提供示範的標準乾淨與受擾動零樣本設定下，從源語言萃取的函數向量是否能引導另一種語言的任務行為。在多種跨語言設定中，應用函數向量大幅提升了表現，這表明函數向量捕捉的是語言無關、任務相關的訊號，而非純粹的語言特定詞彙模式，並凸顯其作為輕量級且可遷移的多語言任務適應機制的潛力。我們觀察到，每個大型語言模型在建構有效的函數向量時，都表現出相對穩定的最佳注意力頭範圍，且此模式在不同語言間保持一致。此外，函數向量能部分重現標準少樣本上下文學習的任務引導效果，同時避免處理多個示範的計算開銷，使其適用於大規模實際應用。我們的程式碼可於 https://github.com/yingjie7/cross_lingual_fvs 取得。

English

Function vectors (FVs) have recently emerged as a promising mechanism for steering the behavior of large language models (LLMs) by injecting task-specific latent direction representations derived from in-context demonstrations. While prior studies have shown that FVs can recover task behavior in structured in-context learning settings, their effectiveness on semantically complex tasks and their ability to generalize across languages remain underexplored. We investigate the cross-lingual transferability of FVs using multilingual multi-label emotion recognition as a challenging semantic classification benchmark. Specifically, we examine whether FVs extracted from a source language can steer task behavior in another language under both standard clean and perturbed zero-shot settings without providing demonstrations during inference. Across diverse cross-lingual settings, applying FVs substantially improves performance, suggesting that FVs capture language-agnostic, task-relevant signals rather than purely language-specific lexical patterns, and highlighting their potential as a lightweight and transferable mechanism for multilingual task adaptation. We observe that each LLM exhibits a relatively stable optimal range of attention heads for constructing effective FVs, and the pattern remains consistent across languages. In addition, FVs can partially replicate the task-steering effects of standard few-shot in-context learning while avoiding the computational overhead of processing multiple demonstrations, making them effective for large-scale practical applications. Our code is available at https://github.com/yingjie7/cross_lingual_fvs.