Husky：一個統一的、開源的語言代理人，用於多步推理

Husky: A Unified, Open-Source Language Agent for Multi-Step Reasoning

June 10, 2024

作者: Joongwon Kim, Bhargavi Paranjape, Tushar Khot, Hannaneh Hajishirzi

cs.AI

摘要

語言代理人透過使用工具來精確執行每個步驟來執行複雜任務。然而，大多數現有的代理人都是基於專有模型或設計來針對特定任務，例如數學或多跳問答。我們介紹了 Husky，一個全面的、開源的語言代理人，它學會在統一的行動空間上進行推理，以應對涉及數值、表格和基於知識的推理的各種複雜任務。Husky在兩個階段之間進行迭代：1) 生成下一步行動以解決給定任務，2) 使用專家模型執行該行動並更新當前解決方案狀態。我們確定了一個全面的行動本體論，用於應對複雜任務，並整理高質量數據來訓練執行這些行動的專家模型。我們的實驗表明，Husky在14個評估數據集上優於先前的語言代理人。此外，我們介紹了 HuskyQA，一個新的評估集，用於對語言代理人進行混合工具推理的壓力測試，重點是檢索缺失知識和進行數值推理。儘管使用了 7B 模型，Husky在這些任務上與甚至超越了前沿的語言模型，如 GPT-4，展示了我們全面方法在應對複雜推理問題上的有效性。我們的代碼和模型可在 https://github.com/agent-husky/Husky-v1 找到。

English

Language agents perform complex tasks by using tools to execute each step precisely. However, most existing agents are based on proprietary models or designed to target specific tasks, such as mathematics or multi-hop question answering. We introduce Husky, a holistic, open-source language agent that learns to reason over a unified action space to address a diverse set of complex tasks involving numerical, tabular, and knowledge-based reasoning. Husky iterates between two stages: 1) generating the next action to take towards solving a given task and 2) executing the action using expert models and updating the current solution state. We identify a thorough ontology of actions for addressing complex tasks and curate high-quality data to train expert models for executing these actions. Our experiments show that Husky outperforms prior language agents across 14 evaluation datasets. Moreover, we introduce HuskyQA, a new evaluation set which stress tests language agents for mixed-tool reasoning, with a focus on retrieving missing knowledge and performing numerical reasoning. Despite using 7B models, Husky matches or even exceeds frontier LMs such as GPT-4 on these tasks, showcasing the efficacy of our holistic approach in addressing complex reasoning problems. Our code and models are available at https://github.com/agent-husky/Husky-v1.