AsyncTool：多任务场景下的异步函数调用能力评估

摘要

基于大语言模型（LLM）的智能体在利用外部工具解决复杂任务方面展现出强大能力。然而，现有评估往往忽略工具使用的时间维度，特别是工具响应延迟的影响，且通常局限于单任务场景。在实际应用中，多个任务常需并发执行，整体效率取决于智能体能否在等待工具响应的空闲时间内高效利用时间。我们将这种能力称为异步工具调用。为评估该能力，我们提出AsyncTool——一个用于评估基于LLM的智能体在具有延迟工具反馈的交互式多任务工具使用环境中的基准测试。AsyncTool同时呈现多个异构任务，并在执行过程中模拟真实的工具响应延迟。通过混合数据进化策略，我们构建了一个多样化的异步多任务数据集，涵盖多种场景和工具使用模式。我们在步骤、子任务和任务三个层级评估模型，并引入面向效率的指标来衡量任务协调与完成效率。大量实验表明，延迟的工具反馈给当前智能体带来重大挑战，并导致性能显著下降。能够更好协调任务切换、依赖追踪和状态维护的模型在AsyncTool上表现更优。我们的分析揭示了当前工具使用智能体的关键失败模式，并为设计未来具有更强时间推理与协调能力的系统提供了实用见解。

English

Large language model (LLM)-based agents have shown strong capabilities in using external tools to solve complex tasks. However, existing evaluations often overlook the temporal dimension of tool use, especially the impact of tool response latency, and are usually limited to single-task settings. In real-world applications, multiple tasks often need to be executed concurrently, and overall efficiency depends on whether an agent can use idle time while waiting for tool responses. We refer to this capability as asynchronous tool calling. To evaluate it, we propose AsyncTool, a benchmark for assessing LLM-based agents in interactive multi-task tool-use environments with delayed tool feedback. AsyncTool presents multiple heterogeneous tasks simultaneously and simulates realistic tool response latency during execution. Using a hybrid data evolution strategy, we construct a diverse asynchronous multitasking dataset that covers multiple scenarios and tool-use patterns. We evaluate models at the step, sub-task, and task levels, and introduce efficiency-oriented metrics to measure task coordination and completion efficiency. Extensive experiments show that delayed tool feedback poses substantial challenges to current agents and leads to clear performance degradation. Models that better coordinate task switching, dependency tracking, and state maintenance achieve stronger performance on AsyncTool. Our analysis identifies key failure modes of current tool-using agents and provides practical insights for designing future systems with stronger temporal reasoning and coordination capabilities.