Aspire：模型能否從模糊目標中自我進化？

摘要

人類學習的許多重要形式始於模糊的目標，例如「成為更優秀的物理學家」或「提升研究能力」。學習者必須解讀目標、辨識能力差距、決定如何學習，並判斷自己是否確實有所進步。相較之下，現有的大型語言模型自我進化研究通常以人類指定的任務與評測指標為起點，把自我進化簡化為最佳化一個明確目標，而非決定要學什麼和怎麼學。我們提出 ASPIRE，一個針對模糊目標驅動之自我進化的基準。ASPIRE 僅提供以自然語言表達的能力目標，下游評測任務則保持隱藏。智能體必須自行選擇資料與更新方法、建構訓練與驗證訊號，並決定何時進行評測，藉此將目標操作化。ASPIRE 在統一的互動式環境中同時支援模型權重進化與智能體框架進化，並以專家撰寫、涵蓋六個目標的 520 個隱藏項目，來評估最終系統。實驗結果顯示，模糊目標會將搜尋努力轉向目標解讀。目前的智能體往往能完成訓練與框架編輯的循環，但權重層面的增益依然稀少且不穩定；最強的進化後框架仍低於工程化的 Qwen-Agent 參考系統。智能體常在不匹配的資料上訓練，並信任狹隘的自我評估，因而局部增益無法遷移至隱藏的評測；持續的搜尋與訓練甚至可能抹除先前的改進。

English

Many important forms of human learning begin with a vague goal, such as "become a better physicist" or "improve at research." Learners must interpret the goal, identify capability gaps, decide how to learn, and determine whether they have actually improved. In contrast, existing work on LLM self-evolution typically begins with tasks and evaluation metrics specified by humans, reducing self-evolution to optimizing an explicit objective rather than deciding what and how to learn. We introduce ASPIRE, a benchmark for vague-goal-driven self-evolution. ASPIRE provides only a natural-language capability goal while downstream evaluation tasks remain hidden. The agent must operationalize the goal by choosing data and update methods, constructing training and validation signals, and deciding when to evaluate. ASPIRE supports both model-weight and agent-harness evolution in a unified interactive environment and evaluates the resulting systems on a hidden, expert-authored set of 520 items spanning six goals. Our experiments show that vague goals redirect search effort toward goal interpretation. Current agents routinely complete training and harness-editing loops, but weight-level gains remain sparse and unstable, and the strongest evolved harness remains below the engineered Qwen-Agent reference. Agents often train on mismatched data and trust narrow self-evaluations, so local gains fail to transfer to hidden evaluation and continued search and training can erase earlier improvements.