ScrambleToolBench：即使自身地图已指向下一步，智能体仍会进行穷尽式搜索

摘要

为了在开放世界环境中稳健运行，自主智能体应当能够仅通过交互来推断陌生系统的行为，即使在缺乏文档的情况下也是如此。然而，现有的工具使用基准在静态环境中暴露语义工具模式，使得智能体可以依赖先验知识而非自主探索。为解决这一局限，我们引入了ScrambleToolBench——一个交互式终端基准，旨在隔离行为推理能力。通过移除语义线索并实施连续任务课程，该基准要求智能体完全通过试错交互来发现隐藏的工具行为。该基准进一步引入了动态挑战，包括映射漂移、随机动作失败和时间执行窗口，以评估智能体是否能够在环境变化时修正和调整其假设。我们对最先进语言模型的评估揭示，成功的最初发现并不能转化为稳健的适应能力。当面临如映射漂移等结构性变化时，智能体无法使用循环追踪等演绎策略，反而表现出信念惯性或退化为穷举搜索。增加测试时推理只会加剧这种代价高昂的暴力搜索，而非实现演绎性恢复。虽然为智能体配备持久记忆可以减少累积错误，但它们仍然无法高效推断结构性变化，凸显了当前智能体推理能力的不足。

English

To operate robustly in open-world environments, autonomous agents should be able to infer the behavior of unfamiliar systems through interaction alone, even in the absence of documentation. However, existing tool-use benchmarks expose semantic tool schemas in static environments, allowing agents to rely on prior knowledge rather than autonomous discovery. To address this limitation, we introduce ScrambleToolBench, an interactive terminal benchmark designed to isolate behavioral reasoning. By removing semantic cues and enforcing a continuous task curriculum, the benchmark requires agents to uncover hidden tool behaviors entirely through trial-and-error interaction. The benchmark further introduces dynamic challenges, including mapping drift, stochastic action failures, and temporal execution windows, to evaluate whether agents can revise and adapt their hypotheses as the environment changes. Our evaluation of state-of-the-art language models reveals that successful initial discovery does not translate into robust adaptation. When faced with structural changes such as mapping drift, agents fail to use deductive strategies such as cycle tracing, and instead exhibit belief inertia or fall back to exhaustive search. Increasing test-time reasoning only amplifies this expensive brute-force search rather than enabling deductive recovery. While equipping agents with persistent memory reduces compounding errors, they remain unable to efficiently infer structural changes, highlighting a gap in current agent reasoning.