學習在改進之前先評估：自動研究代理的評分量規自動歸納

摘要

自主科學研究智慧體越來越常被應用於端對端科學工作流程，包括文獻回顧、資料分析、實驗與報告生成。然而，開放式研究任務往往未能清楚界定完成任務所需的分析、方法與成功標準。因此，智慧體可能遺漏重要分析、使用不當方法，或得出缺乏充分證據支持的結論。為了解決此問題，我們提出 AutoSciRub，這是一個評估優先框架，在研究執行前生成任務特定的可執行評分標準，並以此引導執行、標準層級的驗證以及迭代式修訂。AutoSciRub 將規範不足的指令分解為原子科學目標，將其奠基於相關文獻與任務可見資料，並綜合出具體、可操作且可驗證的標準。由此產生的評分標準將隱含的實驗與證據需求明確化，為實驗與分析提供指引。在修訂階段，以評分標準引導的驗證能識別未達成的標準，並對研究報告及其支援材料進行有針對性的改進。在 ResearchClawBench 上，AutoSciRub 一致地提升了所有受測配置：在固定 Codex 執行框架下，三個骨幹大型語言模型的平均增益為 2.08 分；在使用固定 DeepSeek-V4-Flash 骨幹的三種智慧體執行框架上，平均增益為 2.95 分。在 AstaBench E2E Discovery 隨機抽樣的 20 個任務子集上，AutoSciRub 在三種智慧體執行框架上進一步取得平均 16.8 分的提升，同時維持或增加了成功完成的任務數量。這些結果顯示，評估優先的引導方式為自主科學研究提供了一種有效且可推廣的控制機制（程式碼：https://github.com/zjunlp/AutoSciRub）。

English

Autonomous scientific research agents are increasingly applied to end-to-end scientific workflows, including literature review, data analysis, experimentation, and report generation. However, open-ended research tasks often do not clearly specify the analyses, methods, and success criteria required to complete the task. As a result, agents may miss important analyses, use inappropriate methods, or draw conclusions that are insufficiently supported by evidence. To address the problem, we present AutoSciRub, an evaluation-first framework that induces a task-specific executable rubric before research execution, and uses it to guide execution, criterion-level verification as well as iterative revision. AutoSciRub decomposes an underspecified instruction into atomic scientific goals, grounds them in relevant literature and task-visible data, and synthesizes specific, actionable, and verifiable criteria. The resulting rubric makes implicit experimental and evidential requirements explicit, providing guidance for experiments and analyses. During revision, rubric-guided verification identifies unmet criteria and enables targeted refinement of the research report and its supporting artifacts. On ResearchClawBench, AutoSciRub consistently improves all tested configurations, with an average gain of 2.08 points across three backbone LLMs under the fixed Codex harness and 2.95 points across three agent harnesses using a fixed DeepSeek-V4-Flash backbone. On a randomly sampled 20-task subset of AstaBench E2E Discovery, AutoSciRub further achieves an average improvement of 16.8 points across three agent harnesses, while maintaining or increasing the number of successfully completed tasks. These results demonstrate that evaluation-first guidance provides an effective and generalizable control mechanism for autonomous scientific research (Code: https://github.com/zjunlp/AutoSciRub).