ComfyMind:基于树形规划与反应式反馈的通用生成框架
ComfyMind: Toward General-Purpose Generation via Tree-Based Planning and Reactive Feedback
May 23, 2025
作者: Litao Guo, Xinli Xu, Luozhou Wang, Jiantao Lin, Jinsong Zhou, Zixin Zhang, Bolan Su, Ying-Cong Chen
cs.AI
摘要
随着生成模型的快速发展,通用生成作为一种统一多模态任务的有前景方法,正受到越来越多的关注。尽管取得了这些进展,现有的开源框架往往仍显脆弱,由于缺乏结构化的工作流规划和执行层面的反馈,难以支持复杂的现实应用。为解决这些局限,我们提出了ComfyMind,一个基于ComfyUI平台构建的协作式AI系统,旨在实现稳健且可扩展的通用生成。ComfyMind引入了两大核心创新:语义工作流接口(SWI),它将底层节点图抽象为用自然语言描述的可调用功能模块,支持高级组合并减少结构错误;以及带有局部反馈执行的搜索树规划机制,它将生成过程建模为层次化决策过程,允许在每个阶段进行自适应修正。这些组件共同提升了复杂生成工作流的稳定性和灵活性。我们在三个公开基准上评估了ComfyMind:ComfyBench、GenEval和Reason-Edit,涵盖了生成、编辑和推理任务。结果显示,ComfyMind持续超越现有开源基线,并取得了与GPT-Image-1相当的性能。ComfyMind为开源通用生成AI系统的发展开辟了一条充满希望的道路。项目页面:https://github.com/LitaoGuo/ComfyMind
English
With the rapid advancement of generative models, general-purpose generation
has gained increasing attention as a promising approach to unify diverse tasks
across modalities within a single system. Despite this progress, existing
open-source frameworks often remain fragile and struggle to support complex
real-world applications due to the lack of structured workflow planning and
execution-level feedback. To address these limitations, we present ComfyMind, a
collaborative AI system designed to enable robust and scalable general-purpose
generation, built on the ComfyUI platform. ComfyMind introduces two core
innovations: Semantic Workflow Interface (SWI) that abstracts low-level node
graphs into callable functional modules described in natural language, enabling
high-level composition and reducing structural errors; Search Tree Planning
mechanism with localized feedback execution, which models generation as a
hierarchical decision process and allows adaptive correction at each stage.
Together, these components improve the stability and flexibility of complex
generative workflows. We evaluate ComfyMind on three public benchmarks:
ComfyBench, GenEval, and Reason-Edit, which span generation, editing, and
reasoning tasks. Results show that ComfyMind consistently outperforms existing
open-source baselines and achieves performance comparable to GPT-Image-1.
ComfyMind paves a promising path for the development of open-source
general-purpose generative AI systems. Project page:
https://github.com/LitaoGuo/ComfyMindSummary
AI-Generated Summary