FlowMind：利用LLMs進行自動工作流生成

FlowMind: Automatic Workflow Generation with LLMs

March 17, 2024

作者: Zhen Zeng, William Watson, Nicole Cho, Saba Rahimi, Shayleen Reynolds, Tucker Balch, Manuela Veloso

cs.AI

摘要

在快速發展的機器人流程自動化（RPA）領域中，已經在自動化重複性流程方面取得了顯著進展，但在需要用戶提出的即興或不可預測任務的情況下，其效果會降低。本文介紹了一種新方法，名為FlowMind，利用大型語言模型（LLMs）如生成預訓練變壓器（GPT）的能力來解決這一限制，並創建自動工作流生成系統。在FlowMind中，我們提出了一個通用的提示配方，用於幫助以可靠的應用程序編程接口（APIs）為基礕的LLM推理。通過這一方法，FlowMind不僅減輕了LLMs中幻覺的常見問題，還消除了LLMs與專有數據或代碼之間的直接交互，從而確保了信息的完整性和保密性 - 這是金融服務中的基石。FlowMind通過呈現自動生成工作流的高層描述進一步簡化了用戶交互，使用戶能夠有效地檢查並提供反饋。我們還介紹了NCEN-QA，這是金融領域的一個新數據集，用於對基金N-CEN報告中的問答任務進行基準測試。我們使用NCEN-QA來評估FlowMind生成的工作流的性能，並與FlowMind的基準和消融變體進行比較。我們展示了FlowMind的成功，提出的講座配方中每個組件的重要性，以及FlowMind中用戶交互和反饋的有效性。

English

The rapidly evolving field of Robotic Process Automation (RPA) has made significant strides in automating repetitive processes, yet its effectiveness diminishes in scenarios requiring spontaneous or unpredictable tasks demanded by users. This paper introduces a novel approach, FlowMind, leveraging the capabilities of Large Language Models (LLMs) such as Generative Pretrained Transformer (GPT), to address this limitation and create an automatic workflow generation system. In FlowMind, we propose a generic prompt recipe for a lecture that helps ground LLM reasoning with reliable Application Programming Interfaces (APIs). With this, FlowMind not only mitigates the common issue of hallucinations in LLMs, but also eliminates direct interaction between LLMs and proprietary data or code, thus ensuring the integrity and confidentiality of information - a cornerstone in financial services. FlowMind further simplifies user interaction by presenting high-level descriptions of auto-generated workflows, enabling users to inspect and provide feedback effectively. We also introduce NCEN-QA, a new dataset in finance for benchmarking question-answering tasks from N-CEN reports on funds. We used NCEN-QA to evaluate the performance of workflows generated by FlowMind against baseline and ablation variants of FlowMind. We demonstrate the success of FlowMind, the importance of each component in the proposed lecture recipe, and the effectiveness of user interaction and feedback in FlowMind.