Chat2Workflow: Um Benchmark para Geração de Fluxos de Trabalho Visuais Executáveis com Linguagem Natural
Chat2Workflow: A Benchmark for Generating Executable Visual Workflows with Natural Language
April 21, 2026
Autores: Yi Zhong, Buqiang Xu, Yijun Wang, Zifei Shan, Shuofei Qiao, Guozhou Zheng, Ningyu Zhang
cs.AI
Resumo
目前，可执行可视化工作流已成为工业实际部署中的主流范式，具有极强的可靠性和可控性。然而在当前实践中，这类工作流几乎完全通过人工工程构建：开发人员需要精心设计工作流、为每个步骤编写提示词，并随着需求变化反复修改逻辑——这使得开发成本高昂、耗时且易出错。为研究大语言模型能否自动化这一多轮交互过程，我们提出了Chat2Workflow基准测试，用于直接从自然语言生成可执行的可视化工作流，并提出了一个鲁棒的智能体框架来缓解循环执行错误。Chat2Workflow基于大量真实业务工作流构建，每个实例的设计都确保生成的工作流能够转换并直接部署到Dify、Coze等实际工作流平台。实验结果表明，虽然最先进的语言模型通常能捕捉高层意图，但在生成正确、稳定且可执行的工作流方面仍存在困难，尤其是在复杂或变化的需求下。尽管我们的智能体框架实现了最高5.34%的错误解决率提升，但存在的现实差距使Chat2Workflow成为推进工业级自动化的重要基础。代码已开源：https://github.com/zjunlp/Chat2Workflow。
English
At present, executable visual workflows have emerged as a mainstream paradigm in real-world industrial deployments, offering strong reliability and controllability. However, in current practice, such workflows are almost entirely constructed through manual engineering: developers must carefully design workflows, write prompts for each step, and repeatedly revise the logic as requirements evolve-making development costly, time-consuming, and error-prone. To study whether large language models can automate this multi-round interaction process, we introduce Chat2Workflow, a benchmark for generating executable visual workflows directly from natural language, and propose a robust agentic framework to mitigate recurrent execution errors. Chat2Workflow is built from a large collection of real-world business workflows, with each instance designed so that the generated workflow can be transformed and directly deployed to practical workflow platforms such as Dify and Coze. Experimental results show that while state-of-the-art language models can often capture high-level intent, they struggle to generate correct, stable, and executable workflows, especially under complex or changing requirements. Although our agentic framework yields up to 5.34% resolve rate gains, the remaining real-world gap positions Chat2Workflow as a foundation for advancing industrial-grade automation. Code is available at https://github.com/zjunlp/Chat2Workflow.