TRACE：基於商業規則推理的課程體系——以知識保留之參數化工具檢索於企業大型語言模型

摘要

參數式檢索透過為每個API分配一個唯一的虛擬標記，並訓練模型透過受限束搜尋產生該標記，從而使大型語言模型（LLM）能夠隱式地檢索工具。ToolSense 指出，這種機制有兩個關鍵缺陷：它在訓練過程中破壞了參數化工具知識，並且其束搜尋解碼速度過慢，無法滿足即時部署的需求。我們提出 TRACE（透過增強式思維鏈與企業規則進行工具檢索），這是一個兩階段的課程機制，用以解決此脫節問題。第一階段重複使用 ToolSense 的多格式記憶化監督式微調（SFT），並以低秩適應（LoRA）植入工具知識。第二階段是我們的核心貢獻：模型被訓練在產生 JSON 格式的工具標記列表之前，先輸出一個思考軌跡，其使用了兩種資料來源——來自 ToolSense 的 RRB（重述-推理-分支）配對，以及針對領域專家制定的商業規則所合成的查詢——兩者均經由推理軌跡增強。此訓練目標保留了第一階段的單選題（MCQ）與問答（QA）探測準確率，同時能在生產級延遲下實現單束貪婪解碼。在一個涵蓋兩個企業產品線、總計超過 8,300 項工具的綜合企業目錄上進行評估，TRACE 的第二階段訓練不僅保留了工具理解能力，更進一步提升：MCQ 準確率相較第一階段提升 3.2 個百分點，QA 探測準確率提升 9 個百分點。在檢索方面，TRACE 在領域 A 達到約 86% 的召回率，在領域 B 達到約 60%——相較於嵌入基準方法的約 27% 與約 52%——且兩者均採用單束貪婪解碼，使其可直接以生產級延遲部署。

English

Parametric retrieval enables LLMs to retrieve tools implicitly by assigning each API a unique virtual token and training the model to generate it via constrained beam search. Toolsense shows that this regime has two critical drawbacks: it destroys parametric tool knowledge during training, and its beam-search decoding is too slow for real-time deployment. We introduce TRACE (Tool Retrieval via Augmented Chain-of-thought and Enterprise rules), a two-stage curriculum that resolves this dissociation. Stage 1 reuses the multi-format memorization SFT from ToolSense to seed tool knowledge with LoRA. Stage 2 is our core contribution: the model is trained to emit a thinking trace before producing a JSON list of tool tokens, using two data sources -- RRB pairs from ToolSense and queries synthesized to target business rules curated by domain experts -- both augmented with reasoning traces. This training objective preserves Stage 1 MCQ and QA probing accuracy while enabling single-beam greedy decoding at production latency. Evaluated on a combined enterprise catalog of 8,300+ tools across two enterprise product lines, TRACE training for Stage 2 not only preserves but improves tool understanding: MCQ accuracy gains +3.2 pp and QA probing gains +9 pp over Stage 1. On retrieval, TRACE achieves ~86% recall on Domain A and ~60% on Domain B -- compared to embedding baseline performance of ~27% & ~52% -- both with single-beam greedy decoding, making it directly deployable at production latency.