Chat2Workflow：基于自然语言生成可执行可视化工作流的基准框架

Chat2Workflow: A Benchmark for Generating Executable Visual Workflows with Natural Language

April 21, 2026

作者: Yi Zhong, Buqiang Xu, Yijun Wang, Zifei Shan, Shuofei Qiao, Guozhou Zheng, Ningyu Zhang

cs.AI

摘要

目前，可执行可视化工作流已成为工业实际部署的主流范式，具有高可靠性与强可控性。然而在当前实践中，这类工作流几乎完全通过人工工程构建：开发者需精心设计流程、为每个步骤编写提示词，并随着需求变更反复修改逻辑——导致开发成本高、周期长且易出错。为研究大语言模型能否自动化这一多轮交互过程，我们提出Chat2Workflow基准测试集，支持从自然语言直接生成可执行可视化工作流，并设计了一种鲁棒的智能体框架以缓解循环执行错误。该基准基于大量真实业务工作流构建，每个实例均支持将生成的工作流转译并直接部署至Dify、Coze等实际工作流平台。实验表明，尽管前沿语言模型常能捕捉高层意图，但在生成正确、稳定且可执行的工作流方面仍存在困难，尤其面对复杂或动态变化的需求时。虽然我们的智能体框架将解决率最高提升5.34%，但现实场景中的剩余差距使Chat2Workflow成为推进工业级自动化研究的重要基石。代码已开源：https://github.com/zjunlp/Chat2Workflow。

English

At present, executable visual workflows have emerged as a mainstream paradigm in real-world industrial deployments, offering strong reliability and controllability. However, in current practice, such workflows are almost entirely constructed through manual engineering: developers must carefully design workflows, write prompts for each step, and repeatedly revise the logic as requirements evolve-making development costly, time-consuming, and error-prone. To study whether large language models can automate this multi-round interaction process, we introduce Chat2Workflow, a benchmark for generating executable visual workflows directly from natural language, and propose a robust agentic framework to mitigate recurrent execution errors. Chat2Workflow is built from a large collection of real-world business workflows, with each instance designed so that the generated workflow can be transformed and directly deployed to practical workflow platforms such as Dify and Coze. Experimental results show that while state-of-the-art language models can often capture high-level intent, they struggle to generate correct, stable, and executable workflows, especially under complex or changing requirements. Although our agentic framework yields up to 5.34% resolve rate gains, the remaining real-world gap positions Chat2Workflow as a foundation for advancing industrial-grade automation. Code is available at https://github.com/zjunlp/Chat2Workflow.