ICLR同行评审与反驳流程的启示

摘要

同行評審是科學出版的基石，在ICLR等頂級機器學習會議中亦是如此。隨著投稿量持續增長，深入理解評審過程的本質與動態對於提升效率、效果及已發表論文質量至關重要。本文針對ICLR 2024與2025年的同行評審過程展開大規模分析，聚焦反駁環節前後的評分變化及審稿人與作者間的互動。我們系統考察了評審分數、作者-審稿人參與度、評審提交的時間規律以及共同審稿人的影響效應。通過量化分析結合基於大語言模型的評審文本與反駁討論分類，我們識別出各評分區間論文的常見優缺點，並發現與分數變化關聯最顯著的反駁策略趨勢。研究結果表明：初始分數與共同審稿人的評分是反駁期間分數變動的最強預測因子，這反映出審稿人間存在一定程度的相互影響。反駁機制對於臨界論文的結果改善具有重要價值，深思熟慮的作者回應能實質性改變審稿人觀點。更廣泛而言，本研究為改進同行評審提供了實證依據，既可指導作者制定有效反駁策略，亦有助學術社群設計更公平高效的評審流程。相關代碼與分數變動數據已開源於：https://github.com/papercopilot/iclr-insights。

English

Peer review is a cornerstone of scientific publishing, including at premier machine learning conferences such as ICLR. As submission volumes increase, understanding the nature and dynamics of the review process is crucial for improving its efficiency, effectiveness, and the quality of published papers. We present a large-scale analysis of the ICLR 2024 and 2025 peer review processes, focusing on before- and after-rebuttal scores and reviewer-author interactions. We examine review scores, author-reviewer engagement, temporal patterns in review submissions, and co-reviewer influence effects. Combining quantitative analyses with LLM-based categorization of review texts and rebuttal discussions, we identify common strengths and weaknesses for each rating group, as well as trends in rebuttal strategies that are most strongly associated with score changes. Our findings show that initial scores and the ratings of co-reviewers are the strongest predictors of score changes during the rebuttal, pointing to a degree of reviewer influence. Rebuttals play a valuable role in improving outcomes for borderline papers, where thoughtful author responses can meaningfully shift reviewer perspectives. More broadly, our study offers evidence-based insights to improve the peer review process, guiding authors on effective rebuttal strategies and helping the community design fairer and more efficient review processes. Our code and score changes data are available at https://github.com/papercopilot/iclr-insights.

ICLR同行评审与反驳流程的启示

Insights from the ICLR Peer Review and Rebuttal Process

摘要

Support