形式化潜在思维：大语言模型中思维表征的四条公理

摘要

我们提出了一个用于大语言模型潜在思维表征的公理评估框架，该框架包含独立于下游基准测试分数的指标，能够揭示基准准确率所掩盖的表征失效问题。现有评估方法将表征质量与模型能力混为一谈，导致无法将失败归因于表征本身而非处理表征的模型。我们形式化了四个功能公理（因果性、最小性、可分离性和稳定性），并为每个公理定义了可量化指标，这些指标直接基于表征计算，且独立于下游准确率。我们针对23个推理任务（如空间推理、事实问答）对开放权重大语言模型进行了审计。研究发现：没有任何候选模型能同时满足全部四个公理；表征能可靠区分任务类型，但无法区分同一任务内的两个问题；表征编码的信息量几乎未超越输入嵌入中已存在的信息。该失效模式在密集模型、推理蒸馏模型和强化学习训练模型系列中具有一致性，表明这一差距是结构性的，而非模型规模或训练过程的属性。

English

We introduce an axiomatic evaluation framework for latent thought representations in LLMs, comprising metrics that are independent of downstream benchmark scores and reveal representational failures that benchmark accuracy masks. Existing evaluations conflate representation quality with model capacity. Therefore, failures cannot be attributed to the representation rather than to the model that processes it. We formalize four functional axioms (Causality, Minimality, Separability, and Stability) and define a quantitative measure for each, computed directly on the representation independently of downstream accuracy. We audit open-weight LLMs across 23 reasoning tasks (e.g., Spatial Reasoning, Factual QA). We find that no candidate satisfies all four axioms simultaneously, that the representations distinguish task type reliably but cannot distinguish between two questions within the same task, and that the representations encode little information beyond what is already present in the input embedding. The failure is consistent across dense, reasoning-distilled, and RL-trained model families, indicating that the gap is structural rather than a property of model size or training procedure.