E-Commerce Bench：评估LLM智能体在长期自主商业运营中的表现

摘要

长期智能体任务不仅仅是跨更多交互轮次的短期任务串联。其动态演变的环境和长期依赖关系要求大语言模型（LLM）在数千步之内持续探索、从经验中学习并调整策略。我们推出了E-Commerce Bench，这是首个将多轮对手方谈判和动态事件整合到长达一年商业运营中的开源基准测试。在365个日历年日中，一个LLM智能体同时运营多家在线商店，研究市场、与供应商谈判采购库存、优化销售策略、履行订单、处理退货并管理现金流，以最大化其年末总资产。为了构建逼真的商家侧运营环境，产品和供应商数据来源于真实的电商平台，而全年持续的促销活动、自然灾害和供应链冲击日历则不断重塑需求。为确保可复现性，市场双方均为确定性的：客户购买和退货遵循固定的需求模型，谈判内核决定供应商的定价、让步和决策，LLM仅用于将其语言化。我们从七个维度（包括年末资产）评估了18个前沿模型，发现没有任何单一模型占据主导地位。GPT-5.6 Sol赚取最多收益，将100,000初始资金增长为1,431,425，但在反欺诈维度上位列18个模型中的第16位，且在运营效率上落后于Fable5。在开放权重模型中，Qwen3.8-Max-Preview以416,252的成绩领先，比GLM 5.2（高）高出38%，并在整个时间跨度内展现出最强的学习能力，在重复订单中逐步压低谈判价格。我们的代码可在 https://github.com/QwenLM/E-CommerceBench 获取。

English

Long-horizon agentic tasks go beyond chaining short tasks over more interaction turns. Their evolving dynamic environments and long-range dependencies require Large Language Models (LLMs) to continually explore, learn from experience, and adapt their policies over thousands of steps. We introduce E-Commerce Bench, the first open-source benchmark that integrates multi-round counterpart negotiation and dynamic events into a year-long business operation. Over a 365-day year, an LLM agent concurrently runs multiple online stores, researching the market, negotiating with suppliers to source inventory, optimizing sales strategies, fulfilling orders, handling returns, and managing cash flow to maximize its end-of-year total assets. To construct a realistic merchant-side operating environment, the product and supplier data are derived from a real e-commerce platform, while a year-long calendar of promotions, natural disasters, and supply-chain shocks continually reshapes demand. For reproducibility, both sides of the market are deterministic: customer purchases and returns follow a fixed demand model, while a negotiation kernel determines supplier pricing, concessions, and decisions, with an LLM used only to verbalize them. We evaluate 18 frontier models across seven dimensions, including year-end assets, and find that no single model dominates. GPT-5.6 Sol earns the most, growing the 100,000 opening stake into 1,431,425, yet it ranks 16th of 18 on fraud avoidance and trails Fable5 in operational efficiency. Among open-weight models, Qwen3.8-Max-Preview leads with 416,252, 38% above GLM 5.2 (high), and achieves the strongest learning over the horizon, progressively bargaining down prices across repeated orders. Our code is available at https://github.com/QwenLM/E-CommerceBench.