智能體安全應是運行時契約

摘要

主流典範將AI安全視為透過RLHF、DPO或憲法式AI在模型訓練階段所注入的屬性。我們主張，對於會執行程式碼、修改檔案、發送訊息及更動資料庫的自動化智慧體而言，此種做法在結構上並不充足。智慧體安全應是由運行框架（harness）強制執行的一項運行時契約，且此契約具有兩個互補面向。預防性面向透過沙箱、權限閘門、輸出過濾器與軌跡監控器，在危險動作發生前加以阻擋。證據性面向則要求可驗證的證明，確認良好動作確實發生，並以測試執行、日誌擷取、檔案差異比對及引用溯源等具體證據作為任務提交的門檻。我們以四條公開證據鏈來支撐此立場，並在補充JSON檔案中釋出行層級協定與資料：一項針對52起已記錄之AI智慧體與LLM安全事故的調查、一項包含31個無爭議核心案例及1個具爭議性說明案例的虛假完成審計、一項針對12個公開智慧體系統與運行框架的軌跡模式審計，以及一項針對NeurIPS、ICML與ICLR 2023-2025年共28,560篇錄取論文之標題層級審計，顯示訓練時與部署時出版量之間存在合併8至12倍的失衡。兩個先前需要強制執行安全的社群——電腦安全與實驗科學——最終皆走向兼具預防性與證據性元素的運行時契約；智慧體AI如今同樣面臨此壓力。我們正式提出「智慧體軌跡模式與證據鏈」，基於標準監控器組合提出組合式閘控命題，並勾勒研究議程。智慧體AI中正確的安全單元是「具可查核證據的軌跡」，而非模型本身。

English

The dominant paradigm treats AI safety as a property to be instilled during model training via RLHF, DPO, or Constitutional AI. We argue this is structurally insufficient for autonomous agents that execute code, mutate files, send messages, and modify databases. Agent safety should be a runtime contract enforced by the harness, and the contract has two complementary faces. The preventive face blocks dangerous actions before they happen via sandboxes, permission gates, output filters, and trajectory monitors. The evidential face requires verifiable proof that good actions actually happened, gating task submission on hard evidence such as test runs, log captures, file diffs, and citation grounding. We ground the position in four lines of public evidence, with row-level protocols and data released in the supplementary JSON files: a survey of 52 documented AI-agent and LLM safety incidents, a false-completion audit with 31 non-contested core cases plus one disputed illustrative case, a trajectory-schema audit of 12 public agent systems and harnesses, and a title-level audit of all 28,560 papers accepted at NeurIPS, ICML, and ICLR 2023-2025 showing a pooled 8-12x imbalance between training-time and deployment-time publication. Two prior communities that needed to enforce safety, computer security and the experimental sciences, converged on runtime contracts with both preventive and evidential elements; agentic AI is now under the same pressure. We formalize an Agent Trajectory Schema and Evidence Chain, state a compositional gating proposition based on standard monitor composition, and outline a research agenda. The right unit of safety in agentic AI is the trajectory-with-checkable-evidence, not the model.