层次化实验型智能体

摘要

大型语言模型（LLMs）正越来越多地用于在现实世界中采取行动并支持人类决策，然而大多数智能体依赖参数化知识、固定的后训练数据、检索或搜索。这种范式在无法仅凭先验知识回答的新型领域和复杂查询中就会失效。例如，了解物理定律本身并不足以让LLMs回答复杂物理系统中的查询或完成长时域任务。为解决这一问题，我们提出了分层实验智能体（HExA），一种通过主动实验进行学习的上下文自改进框架。HExA迭代地设计并优化与查询相关的实验，从经验中学习可组合技能的可重用库，并整合实验证据来回答查询或采取行动。HExA无需训练，与任何黑盒模型兼容，且不依赖外部监督、预言机制或离线数据。为评估主动实验能力，我们引入了Interphyre，这是一个基于PHYRE 2D程序化物理环境的工具调用基准，智能体通过模拟API提出干预措施并测试假设。实验表明，当前LLM智能体在此类设置中表现不佳，尤其是在Interphyre的最高难度级别上。Claude Sonnet 4.6的成功率仅为2%，而HExA将同一模型提升至高达77%的成功率。HExA也提升了开放权重模型的表现，并优于ReAct和Reflexion等智能体基线方法。此外，仅使用从较简单级别学习并迁移（无需主动实验）的技能，HExA即可达到44%的成功率，展示了其习得技能的可重用性与泛化能力。总体而言，HExA表明通过主动实验进行学习能够帮助智能体发现有用知识、获取可重用技能，并在陌生的长时域任务中取得高效进展。

English

Large language models (LLMs) are increasingly used to take actions in the real world and support human decision-making, yet most agents rely on parametric knowledge, fixed post-training data, retrieval, or search. This paradigm breaks down in novel domains and for sophisticated queries that cannot be answered from prior knowledge alone. Knowing the laws of physics, for instance, does not by itself enable LLMs to answer queries or complete long-horizon tasks in a complex physical system. To address this, we introduce Hierarchical Experimentalist Agents (HExA), an in-context self-improvement framework to learn from active experimentation. HExA iteratively designs and refines query-relevant experiments, learns a reusable library of composable skills from experience, and integrates experimental evidence to answer queries or take actions. HExA is training-free, compatible with any black-box model, and does not require external supervision, oracles, or offline data. To evaluate active experimentation, we introduce Interphyre, a tool-calling benchmark built on the PHYRE 2D procedural physics environment, where agents propose interventions and test hypotheses through simulation APIs. Experiments show that current LLM agents struggle in these settings, especially on the hardest levels of Interphyre. Claude Sonnet 4.6 achieves only 2% success, while HExA improves the same model to up to 77% success. HExA also improves open-weight models and outperforms agentic baselines such as ReAct and Reflexion. Moreover, using only skills learned from easier levels and transferred without active experimentation, HExA achieves 44% success, demonstrating the reusability and generalization of its learned skills. Overall, HExA shows that learning through active experimentation can help agents discover useful knowledge, acquire reusable skills, and make efficient progress on novel long-horizon tasks.