PIPE-Cypher：面向文本至Cypher系統的自動化企業級基準生成

摘要

企業屬性圖在模式結構、內部術語、領域假設、治理限制以及使用者互動模式上均有極大差異。因此，一個與部署相關的Text2Cypher基準測試，必須反映使用者與代理實際對該圖提出的問題與查詢。建立此類基準測試相當困難，因為模式與數值具有獨特性，且圖結構會隨時間變化。每個自然語言-查詢配對必須可執行、使用真實圖實體、維持多樣性，並在查詢類型與難度層級上保持平衡。我們提出PIPE-Cypher，這是一套本地端基準生成流程，能將運作中的屬性圖與可選的種子查詢（來自客戶問題、分析人員日誌或代理工具呼叫）轉換為平衡的自然語言轉Cypher基準。PIPE-Cypher結合了模式剖析、反向查詢基礎化、受限生成、確定性Cypher治理、執行驗證、編輯處理、多樣性控制，以及經過校準的本地LLM評判器。透過本地Qwen3.5-9B進行生成與評判，PIPE-Cypher輸出3,000個經接受的FinBench/SNB範例，完成三組經審核的消融實驗套件，以人類標籤校準評判器行為，並評估11個本地下游模型。所產生的基準測試刻意具有判別性：零樣本遷移表現薄弱，而少樣本控制則顯示，模式專屬的範例庫能協助相容的模型系列。整體而言，PIPE-Cypher讓Text2Cypher基準測試成為一個可重複的流程，並隨著圖本身、其使用者及其目標工作負載而演進。

English

Enterprise property graphs vary widely in schema structure, internal terminology, domain assumptions, governance constraints, and user interaction patterns. A deployment-relevant Text2Cypher benchmark therefore reflects the questions users and agents actually ask of that graph. Creating such a benchmark is difficult because schemas and values are unique, and graph structure changes over time. Each NL-query pair must also be executable, use real graph entities, preserve diversity, and remain balanced across query types and difficulty levels. We present PIPE-Cypher, a local benchmark-generation pipeline that turns a live property graph and optional seed queries from customer questions, analyst logs, or agent tool calls into balanced NL-to-Cypher benchmarks. PIPE-Cypher combines schema profiling, reverse-query grounding, constrained generation, deterministic Cypher governance, execution validation, redaction, diversity controls, and a calibrated local LLM judge. Using local Qwen3.5-9B generation and judging, PIPE-Cypher exports 3,000 accepted FinBench/SNB examples, completes three audited ablation suites, calibrates judge behavior with human labels, and evaluates 11 local downstream models. The resulting benchmark is deliberately discriminative: zero-shot transfer is weak, while a few-shot control shows that schema-specific example banks can help compatible model families. Together, PIPE-Cypher makes Text2Cypher benchmarking a repeatable process that evolves with the graph, its users, and its target workloads.