SpecFirst：將行為規範提取作為基於代理的從頭程式合成中的首要步驟

摘要

基於LLM的智能體在軟體工程任務中表現優異，尤其是當現有程式碼庫提供上下文時，但從零開始建構程式本質上仍困難得多。近期如ProgramBench的基準測試量化了此差距：僅給定自然語言文件與僅可執行二進位檔作為行為預言，即使是前沿模型也無法解決超過1%的實例。現有框架將文件閱讀、行為探索與程式碼合成混為單一階段，導致智能體探查不足、行為意圖因上下文漂移而流失，並將早期誤解傳遞至最終實作。受經典需求工程啟發，我們主張應將行為規範獲取作為實作前的首要階段。我們提出SpecFirst，一個兩階段框架，強制在程式碼合成前進行規範獲取。專用的規範智能體首先探查二進位檔，將觀察結果與文件結合成結構化規範。接著，程式碼合成智能體利用此規範驅動實作。這種分解在編碼開始前澄清文件歧義，並在合成過程中提供穩定的行為參考。我們在200個ProgramBench實例上評估SpecFirst，涵蓋兩個系列的四種模型，能力跨數量級。SpecFirst始終優於單循環基線，測試通過率提升6.9%–21.3%，二進位檔探查覆蓋率提升9.4%–18.5%，均具統計顯著性。程式碼合成的行為分析進一步顯示，預先制定的規範能促成更早且更持續的程式碼建構。我們的結果證明，顯式需求工程階段是從零開始程式建構的有效範式。

English

LLM-based agents excel at software engineering tasks where an existing codebase provides context, but constructing a program from scratch remains fundamentally harder. Recent benchmarks such as ProgramBench quantify this gap: given only natural-language documentation and an execute-only binary as a behavioral oracle, even frontier models solve fewer than 1% of instances. Existing frameworks conflate documentation reading, behavioral exploration, and code synthesis into a single pass, causing agents to probe insufficiently, lose behavioral intent as context drifts, and propagate early misinterpretations into the final implementation. Inspired by classical requirements engineering, we argue that behavioral specification elicitation should be a first-class phase that precedes implementation. We present SpecFirst, a two-stage framework that forces the specification elicitation before code synthesis. A dedicated spec agent first probes the binary and combines observations with documentation into a structured specification. Next, a code synthesis agent then uses this specification to drive implementation. This decomposition resolves documentation ambiguities before coding begins and provides a stable behavioral reference throughout synthesis. We evaluate SpecFirst on all 200 ProgramBench instances across four models spanning two families and an order of magnitude of capability. SpecFirst consistently outperforms the single-loop baseline, improving test pass rates by 6.9%-21.3% and binary exploration coverage by 9.4%-18.5%, all statistically significant. Behavioral analysis on code synthesis further shows that a prior specification enables earlier and more sustained code construction. Our results demonstrate that an explicit requirements-engineering phase is an effective paradigm for from-scratch program construction.