文档抽取中有效的逐字段选择性风险控制：三种失效模式、有效性阶梯与条件化的收益

摘要

逐字段接受/审查——其选择性风险至多 alpha，即仅当被接受字段的错误率受到控制时才接受该字段——是文档抽取系统所需的信任契约；而自然流程在真实文档上会静默地违反这一契约。在来自 800 份 CORD 收据的 13,859 个真实 claude-sonnet-5 字段（49.0% 正确）上，我们诊断出三种失效模式：文档聚类（设计效应 1.84–2.45）、得分重拟合泄漏（风险 0.127 时覆盖率 0.416，在 95% 的划分中违反 alpha=0.10）、以及并列分数堆积病态（退化分数使阈值网格坍缩，0.030 到 0.001）。我们将修复方案组织成一个有效性阶梯，每一层都说明其保证形式。拟合/验证划分流程为学习融合恢复了期望选择性风险控制：在名义 alpha=0.10 下，风险 0.096 时覆盖率为 0.318，无容差带（生产变体为 0.326）——这是一个平均意义上的点，其实际风险在 47.5% 的重新划分中超过 alpha，并非保证。采用精确二项分布尾部的 Mondrian Learn-then-Test 产生逐组 PAC 保证：字段 iid 覆盖率 0.171（风险 0.068），聚类校正覆盖率 0.140，文档 iid 覆盖率 0.060——这是唯一与文档结构匹配的层级，但坦率地说目前近乎空洞。Support-bin（预先指定的来源分类法）在 Sonnet CORD 采集数据上的每个严谨性层级都胜出（p<1e-4，经 Bonferroni 校正）——这一胜出在使用 Haiku 或 Qwen 处理相同文档时无法复现；而在准确率更高的语料库上，合并阈值方法胜出：条件化恰恰在合并方法无法提供认证之处发挥作用，在其他地方则被学习得分所涵盖。在未参与选择的 claude-haiku-4-5 上进行的冻结配置确认在两个风险水平上均成立；一项由三名标注者进行的盲法人工金标准审计验证了实用层级的接受集风险为 1.3%，而其风险预算为 10%（Fleiss' κ=0.83；标注错误呈单边悲观倾向）。相关代码以 Apache-2.0 许可证发布，并附带种子固定、回归门控的流程。

English

Per-field accept/review with selective risk at most alpha -- accept a field only if the error rate among accepted fields is controlled -- is the trust contract document-extraction systems need, and the natural procedure silently violates it on real documents. On 13,859 genuine claude-sonnet-5 fields from 800 CORD receipts (49.0% correct) we diagnose three failure modes: document clustering (design effect 1.84-2.45), score-refit leakage (coverage 0.416 at risk 0.127, violating alpha=0.10 in 95% of splits), and a tie-mass pathology (a degenerate score collapses the threshold grid, 0.030 to 0.001). We organize the fixes as a validity ladder, guarantee form stated per tier. A fit/val split protocol restores expected-selective-risk control for a learned fusion: coverage 0.318 at risk 0.096 at nominal alpha=0.10, no tolerance band (production variant 0.326) -- an on-average point whose realized risk exceeds alpha in 47.5% of resplits, not a certificate. Mondrian Learn-then-Test with exact binomial tails yields per-group PAC certificates: field-iid 0.171 at risk 0.068, cluster-corrected 0.140, doc-iid 0.060 -- the only tier matching documents, honestly near-vacuous today. Support-bin, the pre-specified provenance taxonomy, wins every rigor tier on the sonnet CORD capture (p<1e-4, Bonferroni-corrected) -- a win that does not replicate on the same documents under haiku or qwen -- while on higher-accuracy corpora pooled thresholds win: conditioning helps exactly where pooled cannot certify, subsumed by a learned score elsewhere. A frozen-configuration confirmation on selection-untouched claude-haiku-4-5 held at both risk levels, and a blind three-annotator human-gold audit verifies the practical tier's accepted-set risk at 1.3% against its 10% budget (Fleiss' kappa=0.83; labels err one-sidedly pessimistic). Released Apache-2.0 with seed-pinned, regression-gated procedures.