超越助手回合：用户回合生成作为语言模型交互意识的探针

Beyond the Assistant Turn: User Turn Generation as a Probe of Interaction Awareness in Language Models

April 3, 2026

作者: Sarath Shekkizhar, Romain Cosentino, Adam Earle

cs.AI

摘要

传统的大语言模型基准测试主要评估助手轮次：模型根据输入生成回复，验证器评判正确性后分析即告结束。这种范式无法衡量LLM是否对其回复后的对话发展具有认知能力。我们提出用户轮生成作为这一空白的探测方法：给定包含用户查询和助手回复的对话上下文，让模型以用户角色生成内容。若模型参数编码了交互意识，所生成的用户轮次应能基于前述上下文作出接地气的延续回应。通过对11个开源权重LLM（Qwen3.5、gpt-oss、GLM等）和5个数据集（数学推理、指令遵循、对话等）的实验表明，交互意识与任务准确性相互解耦。以Qwen3.5系列为例，GSM8K准确率从41%（0.8B）提升至96.8%（397B-A17B），但确定性生成下的真实延续率仍接近零；而采用更高温度采样时，交互意识呈潜伏态显现，延续率可达22%。受控扰动实验验证了该探测方法确实衡量了模型的真实属性，对Qwen3.5-2B进行协作导向的后训练则使延续率提升。我们的结果表明，用户轮生成捕捉到了LLM行为的新维度——交互意识，这一维度在当前仅关注助手表现的基准测试中尚未被探索且不可见。

English

Standard LLM benchmarks evaluate the assistant turn: the model generates a response to an input, a verifier scores correctness, and the analysis ends. This paradigm leaves unmeasured whether the LLM encodes any awareness of what follows the assistant response. We propose user-turn generation as a probe of this gap: given a conversation context of user query and assistant response, we let a model generate under the user role. If the model's weights encode interaction awareness, the generated user turn will be a grounded follow-up that reacts to the preceding context. Through experiments across 11 open-weight LLMs (Qwen3.5, gpt-oss, GLM) and 5 datasets (math reasoning, instruction following, conversation), we show that interaction awareness is decoupled from task accuracy. In particular, within the Qwen3.5 family, GSM8K accuracy scales from 41% (0.8B) to 96.8% (397B-A17B), yet genuine follow-up rates under deterministic generation remain near zero. In contrast, higher temperature sampling reveals interaction awareness is latent with follow up rates reaching 22%. Controlled perturbations validate that the proposed probe measures a real property of the model, and collaboration-oriented post-training on Qwen3.5-2B demonstrates an increase in follow-up rates. Our results show that user-turn generation captures a dimension of LLM behavior, interaction awareness, that is unexplored and invisible with current assistant-only benchmarks.