ChatPaper.aiChatPaper

EdgeBench：揭示从真实世界环境中学习的缩放定律

EdgeBench: Unveiling Scaling Laws of Learning from Real-World Environments

July 6, 2026
作者: Deyao Zhu, Xin Zhou, Shengling Qin, Xuekai Zhu, Hangliang Ding, Shu Zhong, Zixin Wen, Zhonglin Xie, Chenhui Gou, Linxuan Ren, Yueyang Wang, Junfeng Zhong, Rui Liu, Tian Gao, Yangguang Lin, Jingyuan Zhang, Maojia Song, Xuan Qi, Jinhong Wu, Chenyang Zhang, Yinzhu Piao, Ziru Niu, Hongbin Lin, Lingxiang Meng, Peng Tang, Chengyao Tang, Shanyu Wu, Huanyu Zheng, Yu Liu, Liya Zhu, He Wang, Ming Ding, Ziyu Wan, Hao Liu, Sibo Wang, Haotian Zhu, Xintian Zhang, Nan Chai, Yipeng Liu, Panhao Lai, Sihang Yuan, Zixin Su, Ge Zhang, Wangchunshu Zhou, Yantao Du, Wenhao Huang, Guang Shi
cs.AI

摘要

预训练规模法则表明，模型能力随数据和计算量的增长呈现出可预测的提升。然而，模型在部署后从真实世界环境中学习的过程，其理解深度仍远未充分。通过分析约38,000小时的智能体与环境交互数据（涵盖134项真实世界任务），我们首次发现，据我们所知，环境学习中的整体性能遵循一条对数S型规模法则，且拟合精度极高，R²达到0.998。跨模型世代的研究还表明，智能体的学习速度大约每三个月翻一番。这一发现源于EdgeBench——一个包含134项真实世界任务的基准平台，这些任务具有超长任务周期，涵盖科学发现、软件工程、组合优化、专业知识工作、形式化数学以及交互式游戏。每项任务在丰富、多层次的反馈机制下，能够支撑至少12小时的连续智能体运行，且均通过大量专家投入构建而成。我们公开发布了其中51项任务及完整的评估框架，以加速推动关于智能体如何从真实世界经验中学习的研究。
English
Pretraining scaling laws reveal that model capability improves predictably with data and compute. But learning from real world environments after deployment remains far less understood. Analyzing roughly 38,000 hours of agent interaction with the environment across 134 real world tasks, we find, to the best of our knowledge, the first evidence that overall performance during environment learning follows a log-sigmoid scaling law with remarkably high precision, reaching R^2 = 0.998. Across model generations, we also find that agent learning speed roughly doubles every three months. This discovery stems from EdgeBench, a suite of 134 real world tasks with ultra-long horizons, spanning scientific discovery, software engineering, combinatorial optimization, professional knowledge work, formal mathematics, and interactive games. Each task sustains at least 12 hours of continuous agent operation under rich, multilevel feedback, and is built through substantial expert effort. We publicly release 51 tasks and our full evaluation framework to accelerate the study of how agents learn from real world experience.