透過非結構化資料的自適應結構化實現符元高效的資料推論代理

摘要

寶貴的資料仍深埋在非結構化來源之中：網頁、報告、合約、申報文件、財報電話會議，以及 PDF 檔案。企業 AI 的一大賭注，是部署可對這類資料進行推理的大型語言模型（LLM）代理，為每位知識工作者回答複雜問題。代理如今已能做到，但成本高得難以負擔。每個問題都得反覆開啟大文件、搜羅散落的證據，動輒消耗上百萬個 token。然而，假使資料早已結構化，同樣的問題便會化簡為成本低廉的資料庫查詢。舉例來說，在 FanOutQA 基準上，若基於理想的預先結構化儲存進行推理，成本可便宜 28 倍；而當問題擴及更多文件時，差距更會拉大至數個數量級。不過，預先將一切結構化並不可行：文件能承載的可能結構，遠多於任何工作負載所會用到的；而有用的結構與文件，也必須等查詢出現後才能確定。為此，我們提出「代理式資料裂解」（agentic data cracking）：此方法在推理本身進行之際，以適應性與推測性的方式，將非結構化資料結構化。它之所以具有適應性，是因為觀察到的查詢會決定結構化的時機與重點；它之所以具有推測性，是因為其結構化範圍超出目前的問題。每當代理開啟文件作答時，一個裂解子代理便會自已載入的上下文中以邊際成本分叉而出，擷取可能服務於日後相關查詢、且有文件根據的結構。隨著時間過去，完全由結構化資料涵蓋的查詢比率逐漸提高；那些查詢無須開啟文件即可獲得解答，因此代理式準確度得以保持，成本則接近 RAG。在 FanOutQA 上，若每個測試問題只額外搭配一個相關問題，裂解法便能在維持準確度的前提下，讓成本下降 53%。代理式資料裂解，是邁向新一代資料基礎設施的第一步——一套位於模型之下的共享底層，讓推理早已付出代價才發掘出的知識得以在此持續累積，從而支撐對非結構化資料的代理式推理。

English

Valuable data remains embedded in unstructured sources: web pages, reports, contracts, filings, earnings calls, and PDFs. The big bet in enterprise AI is deploying LLM agents that reason over this data to answer complex questions for every knowledge worker. Agents can do this today, but at prohibitive cost. Each question repeatedly opens large documents to recover scattered evidence, consuming up to a million tokens. However, if the data were already structured, the same question would reduce to a cheap database lookup. For example, on FanOutQA benchmark, reasoning over an ideal pre-structured store is 28X cheaper, and the gap grows to orders of magnitude as questions fan out over more documents. Yet structuring everything in advance is not viable: documents hold vastly more possible structure than any workload will use, and the useful structure and documents are unknown until queries arrive. We propose agentic data cracking, a method that structures unstructured data adaptively and speculatively as a byproduct of reasoning itself. Structuring is adaptive because observed queries decide when it happens and what matters, and speculative because it goes beyond the current question. Whenever the agent opens a document to answer, a cracking sub-agent forks from the already-loaded context at marginal cost and extracts grounded structure likely to serve related future queries. Over time, an increasing share of queries is fully covered by structured data and answered without opening a document, keeping agentic accuracy at close to RAG cost. On FanOutQA, extended with merely one related question per test question, cracking cuts cost by 53% while preserving accuracy. Agentic data cracking is a first step toward next-generation data infrastructure for agentic reasoning over unstructured data: a shared substrate beneath the model where knowledge that reasoning already paid to uncover accumulates.