ICLR同行评审与反驳环节的启示

摘要

同行评审是科学出版的基石，在ICLR等顶级机器学习会议中尤为重要。随着投稿量持续增长，深入理解评审过程的本质与动态对提升效率、优化效果及保障论文质量至关重要。本研究对ICLR 2024与2025年的大规模评审数据展开分析，聚焦反驳环节前后的评分变化及审稿人与作者互动模式。通过量化分析结合基于大语言模型的评审意见与反驳讨论文本分类，我们系统考察了评分分布、作者-审稿人参与度、评审提交时间规律以及共同审稿人影响力效应。研究发现：初始评分与共同审稿人评价是反驳期间分数变动的关键预测因子，表明审稿人之间存在相互影响；反驳环节对临界评分论文具有显著积极作用，深思熟虑的作者回应能有效扭转审稿人观点。更广泛而言，本研究为改进同行评审机制提供了实证依据，既指导作者制定有效反驳策略，也助力学术社区设计更公平高效的评审体系。相关代码与评分变化数据已开源：https://github.com/papercopilot/iclr-insights。

English

Peer review is a cornerstone of scientific publishing, including at premier machine learning conferences such as ICLR. As submission volumes increase, understanding the nature and dynamics of the review process is crucial for improving its efficiency, effectiveness, and the quality of published papers. We present a large-scale analysis of the ICLR 2024 and 2025 peer review processes, focusing on before- and after-rebuttal scores and reviewer-author interactions. We examine review scores, author-reviewer engagement, temporal patterns in review submissions, and co-reviewer influence effects. Combining quantitative analyses with LLM-based categorization of review texts and rebuttal discussions, we identify common strengths and weaknesses for each rating group, as well as trends in rebuttal strategies that are most strongly associated with score changes. Our findings show that initial scores and the ratings of co-reviewers are the strongest predictors of score changes during the rebuttal, pointing to a degree of reviewer influence. Rebuttals play a valuable role in improving outcomes for borderline papers, where thoughtful author responses can meaningfully shift reviewer perspectives. More broadly, our study offers evidence-based insights to improve the peer review process, guiding authors on effective rebuttal strategies and helping the community design fairer and more efficient review processes. Our code and score changes data are available at https://github.com/papercopilot/iclr-insights.

ICLR同行评审与反驳环节的启示

Insights from the ICLR Peer Review and Rebuttal Process

摘要

Support