DataFlow-Harness：一個基於真實執行環境的代碼代理平台，用於構建可編輯的LLM數據管道

摘要

大型語言模型（LLMs）正日益被用於自動化資料處理工作流程，然而編碼代理通常產生的腳本並不會自動具體化為持久、可編輯的平台工件。我們將這種斷裂稱為 NL2Pipeline 差距。為彌合此差距，我們引入了 DataFlow-Harness，這是一個引導 LLM 代理透過型別化增量變異（而非自由形式腳本）來構建平台原生有向無環圖（DAG）的平台。該平台結合了用於程序性指導的 DataFlow-Skills、一個暴露即時運算符註冊表與當前管線狀態的模型上下文協議（MCP）層，以及能將對話式創作與視覺化 DAG 編輯器同步的 DataFlow-WebUI。在一個包含 12 項任務的數據工程基準測試中，DataFlow-Harness 達到了 93.3% 的觀察端到端通過率。相較於 Vanilla Claude Code，其測量貨幣成本降低了 72.5%，生成延遲降低了 49.9%；其觀察通過率與 Context-Aware Claude Code 基準相差僅 0.9 個百分點，而成本卻低了 42.8%。逐任務分析表明，Skills 在建構依賴隱含程序性知識時最為有效。這些結果顯示，即時平台接地能夠生成持久、可編輯的工作流程工件，其觀察可靠性接近腳本生成基準，同時建構成本和延遲更低。

English

Large language models (LLMs) are increasingly used to automate data-processing workflows, yet coding agents typically produce scripts that are not automatically materialized as persistent, editable platform artifacts. We call this disconnect the NL2Pipeline gap. To bridge it, we introduce DataFlow-Harness, a platform that guides an LLM agent to construct platform-native directed acyclic graphs (DAGs) through typed, incremental mutations rather than free-form scripts. The platform combines DataFlow-Skills for procedural guidance, a Model Context Protocol (MCP) layer that exposes the live operator registry and current pipeline state, and DataFlow-WebUI, which synchronizes conversational authoring with a visual DAG editor. On a 12-task data-engineering benchmark, DataFlow-Harness achieves a 93.3\% observed end-to-end pass rate. Relative to Vanilla Claude Code, it reduces measured monetary cost by 72.5\% and generation latency by 49.9\%; its observed pass rate is within 0.9 percentage points of the Context-Aware Claude Code baseline while its cost is 42.8\% lower. Per-task analysis indicates that Skills are most useful when construction depends on implicit procedural knowledge. These results show that live platform grounding can produce persistent, editable workflow artifacts with an observed reliability close to script-generation baselines and with lower measured construction cost and latency.