这个引文是否恰当？

摘要

2023年，纽约州一名法官在Mata v. Avianca案中对两名律师予以制裁，原因在于他们提交的法律文书包含由ChatGPT生成的幻觉引用。此类失误大多可通过数据库检索发现；更棘手的问题在于识别那些指向真实案件、却并不支持其所援引命题的引用——而现有针对法律场景的LLM评估在很大程度上忽视了这一失败模式。在本文中，我们通过受控扰动来研究命题级引用支持验证：对从两个法律语料库中获取的真实法律引用进行扰动，要么替换所引案件，要么在同一案件中仅改变精确页码。我们在所得样本上评估了十四个模型配置。对于替换案件的扰动，模型能识别出93%–100%。对于法院意见中的错误精确页码扰动，模型仅能识别37%–61%；在法律文书中则为52%–83%。当模型未能识别错误精确页码扰动时，它们往往是基于主题重叠而非页码级支持来接受该引用。模型规模与扩展推理缩小了这一差距，但并未弥合：GPT-5.4在高强度推理下仍会遗漏法院意见中40%的精确页码错配，以及法律文书中18%的此类错配。提示模型在引用页码处验证支持可提升召回率，但同时也会提高误报率。识别正确的法律主题与验证所引命题的支持是两种不同的能力，而当前模型将二者混为一谈。

English

In 2023, a New York judge sanctioned two attorneys in Mata v. Avianca for filing a brief with hallucinated citations generated by ChatGPT. Such failures are largely caught by database lookups; the harder problem is detecting citations that point to real cases but do not support the propositions for which they are offered -- a failure mode that existing evaluations of LLMs for legal use cases largely overlook. In this paper, we study proposition-level citation support verification through controlled perturbations of real legal citations obtained from two legal corpora, either replacing the cited case or changing only the pinpoint page within the same case. We evaluate fourteen model configurations on the resulting examples. Models catch 93-100% of wrong-case corruptions. They catch only 37-61% of wrong-pinpoint corruptions on court opinions and 52-83% on legal briefs. When models fail to catch wrong-pinpoint corruptions, they accept the citation based on topical overlap rather than page-level support. Scale and extended reasoning narrow the gap but do not close it: GPT-5.4 with high reasoning effort still misses 40% of pinpoint mismatches on court opinions and 18% on briefs. Prompting the model to verify support at the cited page improves recall, but it also raises the false positive rate. Recognizing the right legal topic and verifying support for the cited proposition are distinct capabilities, and current models conflate them.