PlotGen：基於多智能體LLM的科學數據可視化通過多模態反饋

PlotGen: Multi-Agent LLM-based Scientific Data Visualization via Multimodal Feedback

February 3, 2025

作者: Kanika Goswami, Puneet Mathur, Ryan Rossi, Franck Dernoncourt

cs.AI

摘要

科學數據可視化對於將原始數據轉換為可理解的視覺表示至關重要，使模式識別、預測和呈現數據驅動的見解成為可能。然而，新手用戶常常因為選擇適當工具和掌握可視化技術的複雜性而遇到困難。大型語言模型（LLMs）最近展示了在協助程式碼生成方面的潛力，儘管它們在準確性方面存在困難並需要迭代調試。在本文中，我們提出了PlotGen，這是一個新穎的多智能體框架，旨在自動化創建精確的科學可視化。PlotGen協調多個基於LLM的智能體，包括一個查詢規劃智能體，將複雜的用戶請求分解為可執行步驟，一個程式碼生成智能體，將偽代碼轉換為可執行的Python程式碼，以及三個檢索反饋智能體 - 數值反饋智能體、詞彙反饋智能體和視覺反饋智能體 - 通過自我反思利用多模態LLMs迭代地改進生成圖表的數據準確性、文本標籤和視覺正確性。大量實驗表明，PlotGen優於強基線，在MatPlotBench數據集上實現了4-6％的改進，從而增強了用戶對LLM生成的可視化的信任，並由於減少了用於處理圖表錯誤的調試時間，提高了新手的生產力。

English

Scientific data visualization is pivotal for transforming raw data into comprehensible visual representations, enabling pattern recognition, forecasting, and the presentation of data-driven insights. However, novice users often face difficulties due to the complexity of selecting appropriate tools and mastering visualization techniques. Large Language Models (LLMs) have recently demonstrated potential in assisting code generation, though they struggle with accuracy and require iterative debugging. In this paper, we propose PlotGen, a novel multi-agent framework aimed at automating the creation of precise scientific visualizations. PlotGen orchestrates multiple LLM-based agents, including a Query Planning Agent that breaks down complex user requests into executable steps, a Code Generation Agent that converts pseudocode into executable Python code, and three retrieval feedback agents - a Numeric Feedback Agent, a Lexical Feedback Agent, and a Visual Feedback Agent - that leverage multimodal LLMs to iteratively refine the data accuracy, textual labels, and visual correctness of generated plots via self-reflection. Extensive experiments show that PlotGen outperforms strong baselines, achieving a 4-6 percent improvement on the MatPlotBench dataset, leading to enhanced user trust in LLM-generated visualizations and improved novice productivity due to a reduction in debugging time needed for plot errors.