成為優秀人工智慧研究代理需具備哪些條件？探討構想多樣性的關鍵作用

摘要

人工智慧研究代理程式有望透過自動化機器學習模型的設計、實施與訓練來加速科學進程。然而該領域仍處於發展初期，驅動代理程式軌跡成敗的關鍵因素尚未被完全理解。本文探討構思多樣性對代理程式效能的影響。首先，我們分析不同模型與代理框架在MLE-bench（評估AI研究代理的知名基準測試）上的運行軌跡。分析結果顯示，不同模型與代理框架會產生不同程度的構思多樣性，且高效能代理程式往往具有更高的構思多樣性。進一步透過控制實驗調節構思多樣性程度，我們證實提高構思多樣性能有效提升代理程式表現。最後，我們超越MLE-bench標準的獎牌評分機制，透過其他評估指標驗證研究結果，證明本發現在不同代理效能指標下依然成立。

English

AI research agents offer the promise to accelerate scientific progress by automating the design, implementation, and training of machine learning models. However, the field is still in its infancy, and the key factors driving the success or failure of agent trajectories are not fully understood. We examine the role that ideation diversity plays in agent performance. First, we analyse agent trajectories on MLE-bench, a well-known benchmark to evaluate AI research agents, across different models and agent scaffolds. Our analysis reveals that different models and agent scaffolds yield varying degrees of ideation diversity, and that higher-performing agents tend to have increased ideation diversity. Further, we run a controlled experiment where we modify the degree of ideation diversity, demonstrating that higher ideation diversity results in stronger performance. Finally, we strengthen our results by examining additional evaluation metrics beyond the standard medal-based scoring of MLE-bench, showing that our findings still hold across other agent performance metrics.

成為優秀人工智慧研究代理需具備哪些條件？探討構想多樣性的關鍵作用

What Does It Take to Be a Good AI Research Agent? Studying the Role of Ideation Diversity

摘要

Support