SurveyX：基於大型語言模型的學術問卷自動化系統

SurveyX: Academic Survey Automation via Large Language Models

February 20, 2025

作者: Xun Liang, Jiawei Yang, Yezhaohui Wang, Chen Tang, Zifan Zheng, Simin Niu, Shichao Song, Hanyu Wang, Bo Tang, Feiyu Xiong, Keming Mao, Zhiyu li

cs.AI

摘要

大型語言模型（LLMs）展現了卓越的理解能力和龐大的知識庫，表明其可作為自動化問卷生成的高效工具。然而，近期關於自動化問卷生成的研究仍受到一些關鍵限制的約束，如有限的上下文窗口、缺乏深入的內容討論以及系統性評估框架的缺失。受人類寫作過程的啟發，我們提出了SurveyX，這是一個高效且組織化的自動化問卷生成系統，將問卷編寫過程分解為兩個階段：準備階段和生成階段。通過創新性地引入在線參考檢索、名為AttributeTree的預處理方法以及再潤色過程，SurveyX顯著提升了問卷編制的效能。實驗評估結果顯示，SurveyX在內容質量（提升0.259）和引用質量（提升1.76）上均優於現有的自動化問卷生成系統，在多個評估維度上接近人類專家的表現。SurveyX生成的問卷示例可在www.surveyx.cn上查看。

English

Large Language Models (LLMs) have demonstrated exceptional comprehension capabilities and a vast knowledge base, suggesting that LLMs can serve as efficient tools for automated survey generation. However, recent research related to automated survey generation remains constrained by some critical limitations like finite context window, lack of in-depth content discussion, and absence of systematic evaluation frameworks. Inspired by human writing processes, we propose SurveyX, an efficient and organized system for automated survey generation that decomposes the survey composing process into two phases: the Preparation and Generation phases. By innovatively introducing online reference retrieval, a pre-processing method called AttributeTree, and a re-polishing process, SurveyX significantly enhances the efficacy of survey composition. Experimental evaluation results show that SurveyX outperforms existing automated survey generation systems in content quality (0.259 improvement) and citation quality (1.76 enhancement), approaching human expert performance across multiple evaluation dimensions. Examples of surveys generated by SurveyX are available on www.surveyx.cn