幂律图注意力:缩放点积注意力的精确泛化,推理阶段的经验坍缩
Power law graph attention: exact generalization of scaled dot-product attention, empirical collapse at inference
August 10, 2026
作者: Burc Gokden
cs.AI
摘要
基于幂律解码器表示的大语言模型(PLDR-LLM)及其注意力机制——幂律图注意力(PLGA)——将缩放点积注意力(SDPA)的固定双线性形式替换为学习到的、由输入生成的双线性算子G_{LM},该算子由正张量A_{LM}通过逐元素幂律构建而成。该架构具有完整的规范说明,并已对照固定的参考发布版本进行了验证;各项断言被标注为定理、条件定理、测量结果或猜想。无条件成立:PLGA在G_{LM}=I时精确包含SDPA;A_{LM}与A_P均为严格逐元素正,且A_{LM}具有Perron-Frobenius结构;DAG正则化器具有NOTEARS游走计数形式,正性阻碍了精确无环性;并且,在非共振条件下(标准旋转频率满足该条件),换位子判据可识别出哪些算子保持相对位置依赖性。推理坍缩定理:演绎输出的精确输入不变性将推理坍缩为具有常算子的广义SDPA。测量不变性:相对波动在10⁻⁶及以下;扰动界量化了缓存推理,但未能对其加以证明;所组装的代理指标未能捕获解码裕度。一个条件性三阶段机制(旋转扭曲、集中、行映射压缩)在已发布的检查点上测得。在全局Gram下进行的分块训练与评分被明确表述为在显式目标暴露下进行;在测试样本上,分块评分与序贯评分选择出相同的答案,且在已发表的TruthfulQA概率质量度量上每个条目的一致程度在5×10⁻⁵以内。自组织临界性作为一个具有内禀序参量的唯象框架被引入;未决断言转变为可证伪的猜想。选定的证明核心已在Lean 4中通过机器验证。
English
The Large Language Model from Power Law Decoder Representations (PLDR-LLM) and its attention, Power Law Graph Attention (PLGA), replace the fixed bilinear form of scaled dot-product attention (SDPA) with a learned, input-generated bilinear operator G_{LM}, built from a positive tensor A_{LM} by elementwise power laws. The architecture is fully specified, verified against pinned reference releases; claims are labeled theorem, conditional theorem, measurement, or conjecture. Unconditionally: PLGA contains SDPA exactly at G_{LM}=I; A_{LM} and A_P are strictly entrywise positive, with Perron-Frobenius structure on A_{LM}; the DAG regularizer has the NOTEARS walk-counting form and positivity obstructs exact acyclicity; and, under nonresonance (satisfied by standard rotary frequencies), a commutant criterion identifies which operators preserve relative-position dependence. An inference-collapse theorem: exact input invariance of deductive outputs collapses inference to generalized SDPA with a constant operator. Measured invariance: relative fluctuations of 10^{-6} and below; perturbation bounds quantify but do not certify cached inference; the assembled proxy misses the decoding margin. A conditional three-stage mechanism (rotary twirl, concentration, row-map contraction) is measured on a released checkpoint. Blockwise training and scoring under the global Gram are stated with explicit target exposure; on tested samples, block and sequential scoring select identical answers and agree on the published TruthfulQA probability-mass metric within 5times 10^{-5} per item. Self-organized criticality enters as a phenomenological framework with an intrinsic order parameter; open claims become falsifiable conjectures. Selected proof cores are machine-checked in Lean 4.