層級式實驗主義智能體

摘要

大型語言模型（LLMs）日益被用於在現實世界中採取行動，並輔助人類決策，然而大多數智能體仍依賴於參數化知識、固定的訓練後資料、檢索或搜尋機制。這種範式在面對新穎領域以及無法僅憑既有知識回答的複雜查詢時，便會失效。例如，即便具備物理定律知識，大型語言模型也無法自行回答關於複雜物理系統的查詢，或完成需要長期規劃的任務。為解決此問題，我們提出「分層實驗性智能體框架」（Hierarchical Experimentalist Agents, HExA），這是一種透過主動實驗進行學習的情境內自我改進框架。HExA能反覆設計並優化與查詢相關的實驗，從經驗中學習可重複使用的組合式技能庫，並整合實驗證據以回答查詢或採取行動。該框架無需訓練，可相容於任何黑箱模型，且不依賴外部監督、預言機制或離線資料。為評估主動實驗能力，我們建立了基於PHYRE二維程序化物理環境的工具調用基準測試Interphyre，其中智能體需透過模擬API提出干預方案並測試假設。實驗顯示，現有的大型語言模型智能體在這類場景中表現不佳，尤其對Interphyre中最困難的關卡。Claude Sonnet 4.6的成功率僅2%，而HExA將同一模型的成功率提升至77%。此外，HExA同樣能提升開放權重模型的表現，並優於ReAct、Reflexion等基於智能體的方法。更進一步，僅使用從較簡單關卡中習得並在無主動實驗情況下遷移的技能，HExA即可達到44%的成功率，證明了其所學技能的可重複使用性與泛化能力。整體而言，HExA表明，透過主動實驗進行學習，能幫助智能體發現有用知識、習得可重複使用的技能，並在新穎的長期任務中取得高效進展。

English

Large language models (LLMs) are increasingly used to take actions in the real world and support human decision-making, yet most agents rely on parametric knowledge, fixed post-training data, retrieval, or search. This paradigm breaks down in novel domains and for sophisticated queries that cannot be answered from prior knowledge alone. Knowing the laws of physics, for instance, does not by itself enable LLMs to answer queries or complete long-horizon tasks in a complex physical system. To address this, we introduce Hierarchical Experimentalist Agents (HExA), an in-context self-improvement framework to learn from active experimentation. HExA iteratively designs and refines query-relevant experiments, learns a reusable library of composable skills from experience, and integrates experimental evidence to answer queries or take actions. HExA is training-free, compatible with any black-box model, and does not require external supervision, oracles, or offline data. To evaluate active experimentation, we introduce Interphyre, a tool-calling benchmark built on the PHYRE 2D procedural physics environment, where agents propose interventions and test hypotheses through simulation APIs. Experiments show that current LLM agents struggle in these settings, especially on the hardest levels of Interphyre. Claude Sonnet 4.6 achieves only 2% success, while HExA improves the same model to up to 77% success. HExA also improves open-weight models and outperforms agentic baselines such as ReAct and Reflexion. Moreover, using only skills learned from easier levels and transferred without active experimentation, HExA achieves 44% success, demonstrating the reusability and generalization of its learned skills. Overall, HExA shows that learning through active experimentation can help agents discover useful knowledge, acquire reusable skills, and make efficient progress on novel long-horizon tasks.