ChatPaper.aiChatPaper

一个冻结的12B模型在验证任务上击败了前沿模型:100%准确率、零token、比特精确、永久。

A Frozen 12B Beats Frontier Models on Verified Work: 100% Accuracy, 0 Tokens, Bit-Exact, Forever

July 26, 2026
作者: Sietse Schelpe
cs.AI

摘要

如今改进语言模型仍意味着重新训练:巨大的算力消耗、每个周期产生一个新的不透明模型、以及非确定性输出。我们选择了一条相反路径:模型保持冻结状态,而在其旁侧不断积累已验证解决方案的持久化记忆。一旦某个问题族被解决并通过了从不查阅答案库的独立验证步骤,该族的每个新实例都能以零生成令牌、比特精确、确定性的方式得到解答。在跨越九大问题族的180个全新实例上,来自四家供应商的四种架构(包含密集模型与混合专家模型)均以每个答案零生成令牌获得180/180的完美得分:执行绑定的能力从此与参数规模解耦。阴性对照实验将全部能力归因于该记忆库——一旦清空记忆,系统便无法解决任何问题。同样的"先验证后存储"协议也适用于开放式推理:四种模型均达成88/88的一致性门控接受率、机器检验的形式化证明,以及77/80的推理方法迁移率。记忆选取耗时1.4微秒;一次完整重用在36毫瓦时功耗下仅需6-23毫秒。在包含4500项已验证条目的记忆库中,近似相似性检索的错误率达94.3%,而精确寻址则完全零错误。该记忆库还能作为工作上下文使用,其规模远超任何已部署引擎:在单块46GB GPU上以扁平内存实现600万令牌的可移动窗口,而vLLM在30,399令牌处停止,SGLang则在超过32,000令牌时静默截断。在公开基准测试中,前沿模型在原始从头推理能力上仍远超任何120亿参数模型;但对于本系统已解决并验证的所有任务,对比结果完全反转:前沿模型API调用每次查询都要执行全新的生成过程,而已验证重用的成本为零令牌,且每次返回完全相同的比特。本报告附带一个提供免费限流访问的公开测试平台:https://corbenic-galahad-bench.hf.space
English
Improving a language model today means retraining it: enormous compute, a new opaque model each cycle, non-deterministic output. We take the opposite path: the model stays frozen, and a persistent memory of verified solutions grows beside it. Once a problem family is solved and has passed an independent verification step that never consults the answer key, every new instance of that family is answered at zero generation tokens, bit-exact, deterministically. Across 180 fresh instances spanning nine problem families, four architectures from four vendors - dense and mixture-of-experts - each score 180/180 at zero generation tokens per answer: execution-bound capability decoupled from parameter scaling. A negative control attributes the capability fully to the memory: emptied, it solves nothing. The same verify-before-store contract holds for open-ended reasoning: 88/88 consistency-gated acceptances across all four models, machine-checked formal proof, and reasoning-method transfer at 77/80. Memory selection takes 1.4 microseconds; a full reuse completes in 6-23 ms at 36 mWh. Approximate similarity retrieval selects the wrong item 94.3% of the time on a 4,500-item verified store where exact addressing makes zero errors. The store also serves as working context at a scale no shipped engine matches: a 6,000,000-token movable window on a single 46 GB GPU at flat memory, where vLLM stops at 30,399 tokens and SGLang silently truncates past 32,000. On published benchmarks, frontier models remain far ahead of any 12B at raw from-scratch reasoning; on everything this system has solved and verified, the comparison inverts: a frontier API call pays a fresh generation pass on every query, forever, while verified reuse costs zero tokens and returns the identical bits every time. A public testbench with free, rate-limited access accompanies this report: https://corbenic-galahad-bench.hf.space