FlowMind：利用LLMs实现自动工作流生成

FlowMind: Automatic Workflow Generation with LLMs

March 17, 2024

作者: Zhen Zeng, William Watson, Nicole Cho, Saba Rahimi, Shayleen Reynolds, Tucker Balch, Manuela Veloso

cs.AI

摘要

机器人流程自动化（RPA）这一快速发展的领域在自动化重复性流程方面取得了显著进展，但在需要用户提出的即兴或不可预测任务的情况下，其效果会减弱。本文介绍了一种新方法FlowMind，利用大型语言模型（LLMs）如生成式预训练变换器（GPT）的能力来解决这一局限，并创建一个自动工作流生成系统。在FlowMind中，我们提出了一个用于讲座的通用提示配方，帮助将LLM的推理与可靠的应用程序编程接口（APIs）联系起来。通过这种方式，FlowMind不仅减轻了LLMs中幻觉的常见问题，还消除了LLMs与专有数据或代码之间的直接交互，从而确保了信息的完整性和保密性 - 这是金融服务中的基石。FlowMind通过展示自动生成工作流的高级描述进一步简化了用户交互，使用户能够有效地检查并提供反馈。我们还介绍了NCEN-QA，这是金融领域的一个新数据集，用于对基金的N-CEN报告进行问答任务的基准测试。我们使用NCEN-QA来评估由FlowMind生成的工作流与FlowMind的基线和消融变体的性能。我们展示了FlowMind的成功，提出的讲座配方中每个组件的重要性，以及用户交互和反馈在FlowMind中的有效性。

English

The rapidly evolving field of Robotic Process Automation (RPA) has made significant strides in automating repetitive processes, yet its effectiveness diminishes in scenarios requiring spontaneous or unpredictable tasks demanded by users. This paper introduces a novel approach, FlowMind, leveraging the capabilities of Large Language Models (LLMs) such as Generative Pretrained Transformer (GPT), to address this limitation and create an automatic workflow generation system. In FlowMind, we propose a generic prompt recipe for a lecture that helps ground LLM reasoning with reliable Application Programming Interfaces (APIs). With this, FlowMind not only mitigates the common issue of hallucinations in LLMs, but also eliminates direct interaction between LLMs and proprietary data or code, thus ensuring the integrity and confidentiality of information - a cornerstone in financial services. FlowMind further simplifies user interaction by presenting high-level descriptions of auto-generated workflows, enabling users to inspect and provide feedback effectively. We also introduce NCEN-QA, a new dataset in finance for benchmarking question-answering tasks from N-CEN reports on funds. We used NCEN-QA to evaluate the performance of workflows generated by FlowMind against baseline and ablation variants of FlowMind. We demonstrate the success of FlowMind, the importance of each component in the proposed lecture recipe, and the effectiveness of user interaction and feedback in FlowMind.