开放世界课程学习中的编程梦想

Dreaming in Code for Curriculum Learning in Open-Ended Worlds

February 9, 2026

作者: Konstantinos Mitsides, Maxence Faldor, Antoine Cully

cs.AI

摘要

开放式学习将智能视为在与不断扩展的环境空间持续互动中涌现的结果。尽管近期研究利用基础模型以编程方式生成多样化环境，但这些方法往往侧重于发现孤立行为而非协调可持续的进阶过程。在复杂的开放世界中，挑战组合空间的庞大规模使得智能体难以发现始终保持可学习性的经验序列。为此，我们提出代码造梦（DiCode）框架，通过基础模型合成可执行环境代码，为能力提升搭建渐进式学习阶梯。在DiCode中，"造梦"表现为具象化世界代码层面的变体。我们在Craftax——一个具有丰富机制和长周期进阶特性的挑战性开放基准测试中实现了DiCode。实验表明，DiCode使智能体能够掌握长周期技能，较最强基线平均回报提升16%，并在现有方法均告失败的后期战斗任务中实现非零成功率。我们的研究结果证明，代码级环境设计为课程控制提供了实用机制，能够构建衔接开放世界能力鸿沟的中间环境。项目页面与源代码详见：https://konstantinosmitsides.github.io/dreaming-in-code 与 https://github.com/konstantinosmitsides/dreaming-in-code。

English

Open-ended learning frames intelligence as emerging from continual interaction with an ever-expanding space of environments. While recent advances have utilized foundation models to programmatically generate diverse environments, these approaches often focus on discovering isolated behaviors rather than orchestrating sustained progression. In complex open-ended worlds, the large combinatorial space of possible challenges makes it difficult for agents to discover sequences of experiences that remain consistently learnable. To address this, we propose Dreaming in Code (DiCode), a framework in which foundation models synthesize executable environment code to scaffold learning toward increasing competence. In DiCode, "dreaming" takes the form of materializing code-level variations of the world. We instantiate DiCode in Craftax, a challenging open-ended benchmark characterized by rich mechanics and long-horizon progression. Empirically, DiCode enables agents to acquire long-horizon skills, achieving a 16% improvement in mean return over the strongest baseline and non-zero success on late-game combat tasks where prior methods fail. Our results suggest that code-level environment design provides a practical mechanism for curriculum control, enabling the construction of intermediate environments that bridge competence gaps in open-ended worlds. Project page and source code are available at https://konstantinosmitsides.github.io/dreaming-in-code and https://github.com/konstantinosmitsides/dreaming-in-code.