以Google論文助手工具邁向科學審查自動化

摘要

人工智慧正推動科學發現的革命，從假說生成到數學定理證明的每個環節都因之加速。然而，這種快速推進也帶來系統性挑戰：傳統的人類同行評審無法擴展以應付AI輔助科學帶來的大量稿件。最終，為了解決這個矛盾，我們必須同樣部署AI來加速驗證與評審流程本身。為了框定這一轉型過程的討論，我們提出一個包含四個漸進層級的分類法，用以描述AI與人類在科學評估中的協作，並探討每個層級涉及的各種取捨。作為邁向該未來的具體步驟，我們推出了論文助手工具（Paper Assistant Tool, PAT），這是一個專為深入科學評審與驗證所設計的智能體AI框架。PAT能完整讀入科學手稿，並產出全面的評估報告，包括檢驗理論結果、驗證實驗、提出改進建議，以及識別潛在缺陷。透過運用推理規模化技術，PAT能比單次模型調用找出更深層的問題，在SPOT基準測試的數學錯誤檢測中，零樣本召回率提升了34%。PAT已在兩個主要計算機科學會議——STOC與ICML——作為作者投稿前的工具進行試點部署，結果顯示它能有效識別關鍵錯誤，並對研究論文提出實質改進建議。透過及早發現錯誤，PAT減輕了審稿人的認知負擔，同時保留其對評審結果的控制權。

English

Artificial intelligence is driving a revolution in scientific discovery, accelerating everything from hypothesis generation to mathematical theorem proving. However, this rapid acceleration is creating a systemic challenge: traditional human peer review cannot scale to match the influx of AI-assisted science. Ultimately, to resolve this tension, we must also deploy AI to accelerate the verification and review process itself. To frame the discussion around this transition, we propose a taxonomy consisting of four progressive levels of AI-human collaboration in scientific evaluation, and discuss various trade-offs involved with each. As a step toward this future, we introduce the Paper Assistant Tool (PAT), an agentic AI framework built for deep scientific review and verification. PAT ingests full scientific manuscripts and produces a comprehensive evaluation, checking theoretical results, validating experiments, suggesting improvements, and identifying potential flaws. By utilizing inference scaling techniques, PAT is able to identify deeper issues than a single model call alone, achieving a 34% improvement over zero-shot recall on mathematical errors in the SPOT benchmark. Pilot deployments of PAT as a pre-submission tool for authors at two major Computer Science conferences -- STOC and ICML -- demonstrate its ability to identify critical errors and suggest substantive improvements to research papers. By catching errors early, PAT eases the cognitive burden placed on referees, while preserving their control over the outcomes of the review process.