开放世界课程学习中的编程梦想

Dreaming in Code for Curriculum Learning in Open-Ended Worlds

February 9, 2026

作者: Konstantinos Mitsides, Maxence Faldor, Antoine Cully

cs.AI

摘要

开放式学习将智能视为在与不断扩展的环境空间持续互动中涌现的产物。尽管近期研究利用基础模型以编程方式生成多样化环境，但这些方法往往侧重于发现孤立行为，而非协调可持续的进阶过程。在复杂的开放式世界中，挑战的组合空间极为庞大，导致智能体难以发现始终保持可学习性的经验序列。为此，我们提出"代码筑梦"（DiCode）框架，通过基础模型合成可执行环境代码，为能力提升搭建阶梯式学习路径。在DiCode中，"筑梦"具体表现为实现代码层面的世界变体。我们在Craftax这一具有丰富机制和长周期进阶特性的开放式基准测试中实例化了DiCode。实验表明，DiCode能使智能体掌握长周期技能，其平均回报率较最强基线提升16%，并在前期方法全部失败的终局战斗任务中实现非零成功率。研究结果证明，代码级环境设计为课程控制提供了实用机制，能够构建衔接开放式世界中能力断层的中间环境。项目页面与源代码详见：https://konstantinosmitsides.github.io/dreaming-in-code 及 https://github.com/konstantinosmitsides/dreaming-in-code。

English

Open-ended learning frames intelligence as emerging from continual interaction with an ever-expanding space of environments. While recent advances have utilized foundation models to programmatically generate diverse environments, these approaches often focus on discovering isolated behaviors rather than orchestrating sustained progression. In complex open-ended worlds, the large combinatorial space of possible challenges makes it difficult for agents to discover sequences of experiences that remain consistently learnable. To address this, we propose Dreaming in Code (DiCode), a framework in which foundation models synthesize executable environment code to scaffold learning toward increasing competence. In DiCode, "dreaming" takes the form of materializing code-level variations of the world. We instantiate DiCode in Craftax, a challenging open-ended benchmark characterized by rich mechanics and long-horizon progression. Empirically, DiCode enables agents to acquire long-horizon skills, achieving a 16% improvement in mean return over the strongest baseline and non-zero success on late-game combat tasks where prior methods fail. Our results suggest that code-level environment design provides a practical mechanism for curriculum control, enabling the construction of intermediate environments that bridge competence gaps in open-ended worlds. Project page and source code are available at https://konstantinosmitsides.github.io/dreaming-in-code and https://github.com/konstantinosmitsides/dreaming-in-code.