Hephaestus：透過持續預訓練來提升大型語言模型的基本代理能力

Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training

February 10, 2025

作者: Yuchen Zhuang, Jingfeng Yang, Haoming Jiang, Xin Liu, Kewei Cheng, Sanket Lokegaonkar, Yifan Gao, Qing Ping, Tianyi Liu, Binxuan Huang, Zheng Li, Zhengyang Wang, Pei Chen, Ruijie Wang, Rongzhi Zhang, Nasser Zalmout, Priyanka Nigam, Bing Yin, Chao Zhang

cs.AI

摘要

由於缺乏以代理為導向的預訓練數據，基於LLM的自主代理通常依賴於複雜的提示或廣泛的微調，這往往無法引入新的能力，同時保留強大的泛化能力。我們介紹了Hephaestus-Forge，這是第一個旨在增強LLM代理在API函數調用、內在推理和規劃以及適應環境反饋方面基本能力的大規模預訓練語料庫。Hephaestus-Forge包括103B代理特定數據，涵蓋76,537個API，包括工具文檔，以介紹API函數知識，以及函數調用軌跡，以加強內在推理。為了探索有效的訓練協議，我們研究了標度律，以確定在數據混合比中的最佳配方。通過在Hephaestus-Forge上持續進行預訓練，Hephaestus在三個代理基準測試中表現優於小型到中型規模的開源LLM，並與商業LLM相媲美，展示了我們的預訓練語料庫在增強基本代理能力和LLM對新任務或環境的泛化能力方面的有效性。

English

Due to the scarcity of agent-oriented pre-training data, LLM-based autonomous agents typically rely on complex prompting or extensive fine-tuning, which often fails to introduce new capabilities while preserving strong generalizability. We introduce Hephaestus-Forge, the first large-scale pre-training corpus designed to enhance the fundamental capabilities of LLM agents in API function calling, intrinsic reasoning and planning, and adapting to environmental feedback. Hephaestus-Forge comprises 103B agent-specific data encompassing 76,537 APIs, including both tool documentation to introduce knowledge of API functions and function calling trajectories to strengthen intrinsic reasoning. To explore effective training protocols, we investigate scaling laws to identify the optimal recipe in data mixing ratios. By continual pre-training on Hephaestus-Forge, Hephaestus outperforms small- to medium-scale open-source LLMs and rivals commercial LLMs on three agent benchmarks, demonstrating the effectiveness of our pre-training corpus in enhancing fundamental agentic capabilities and generalization of LLMs to new tasks or environments.