E-Commerce Bench：評估LLM智能體於長時程自主商業營運之表現

摘要

長時間跨度的代理任務並不止於在更多互動回合中串接短期任務。其持續演變的動態環境與長程依賴，要求大型語言模型（LLMs）在數千個步驟中持續探索、從經驗中學習，並調整其策略。我們提出 E-Commerce Bench，這是第一個將多輪交易對手談判與動態事件整合為一年期商業營運的開源基準。在365天的年度內，一個 LLM 代理同時經營多家線上商店：研究市場、與供應商談判以取得庫存、最佳化銷售策略、履行訂單、處理退貨，並管理現金流，以最大化年終總資產。為了建構逼真的商家端營運環境，產品與供應商資料皆來自真實的電子商務平台，同時一份橫跨一年的促銷、天然災害與供應鏈衝擊事件日曆持續重塑需求。為確保可重現性，市場的兩端均採確定性設計：顧客購買與退貨遵循固定需求模型，而談判核心決定供應商的定價、讓步與決策，LLM 僅被用來將這些結果轉化為語言。我們在七個維度上評估了18個前沿模型，包含年終資產，結果發現沒有任何單一模型全面主導。GPT-5.6 Sol 賺得最多，將100,000的初始資本增長至1,431,425，但在詐騙規避上排名18個模型中的第16位，且營運效率落後於 Fable5。在開放權重模型中，Qwen3.8-Max-Preview 以416,252領先，比 GLM 5.2 (high) 高出38%，並在整個營運期間展現最強的學習能力，能在重複訂單中逐步議價壓低價格。我們的程式碼已公開於 https://github.com/QwenLM/E-CommerceBench。

English

Long-horizon agentic tasks go beyond chaining short tasks over more interaction turns. Their evolving dynamic environments and long-range dependencies require Large Language Models (LLMs) to continually explore, learn from experience, and adapt their policies over thousands of steps. We introduce E-Commerce Bench, the first open-source benchmark that integrates multi-round counterpart negotiation and dynamic events into a year-long business operation. Over a 365-day year, an LLM agent concurrently runs multiple online stores, researching the market, negotiating with suppliers to source inventory, optimizing sales strategies, fulfilling orders, handling returns, and managing cash flow to maximize its end-of-year total assets. To construct a realistic merchant-side operating environment, the product and supplier data are derived from a real e-commerce platform, while a year-long calendar of promotions, natural disasters, and supply-chain shocks continually reshapes demand. For reproducibility, both sides of the market are deterministic: customer purchases and returns follow a fixed demand model, while a negotiation kernel determines supplier pricing, concessions, and decisions, with an LLM used only to verbalize them. We evaluate 18 frontier models across seven dimensions, including year-end assets, and find that no single model dominates. GPT-5.6 Sol earns the most, growing the 100,000 opening stake into 1,431,425, yet it ranks 16th of 18 on fraud avoidance and trails Fable5 in operational efficiency. Among open-weight models, Qwen3.8-Max-Preview leads with 416,252, 38% above GLM 5.2 (high), and achieves the strongest learning over the horizon, progressively bargaining down prices across repeated orders. Our code is available at https://github.com/QwenLM/E-CommerceBench.