免梯度适应：仿射统计量传输及其证书的启示

摘要

测试时自适应（Test-time adaptation, TTA）通常假设模型参数可以在推理时更新。对于仅支持推理的加速器、冻结模型或第三方模型以及内存受限的部署而言，这一假设具有很强的限制性；同时，基于批归一化（BatchNorm）的标准TTA配置在没有批归一化的架构上也可能失效。本文研究已学习模型必须保持冻结时的适应问题。我们提出CASTER，一种无梯度方法：它在判别性子空间中存储源类别统计量，根据目标批次矩估计类别共享的仿射变换，并在分类前解析地迁移源类别分布。CASTER不需要反向传播、优化器状态或存储的源特征库。在四个骨干网络和七个数据集上，CASTER在28个骨干网络-数据集设置中的27个上优于使用相同冻结特征的k-NN，同时其中位状态量仅为k-NN的1/18。仿射迁移并非总是可靠。在ImageNet-C上，批次仅含64个样本而类别数为1000，无条件迁移导致top-1准确率下降21.2个百分点。因此，我们提出一种经验性的残差-间隔可迁移性证书。在307个评估单元中，所有损失超过10个百分点的迁移，其证书值均高于3.9，尽管良性情形与破坏性情形并未被完全区分开。门控将无条件迁移平均-3.35个百分点的效应转化为+1.69个百分点的收益，并且在很宽的阈值范围内，性能与最佳阈值下的结果相差不超过0.3个百分点。最后，我们证明该证书具有机制特异性：将其应用于Tent时，它只接受4.3%的更新，并仅保留Tent可用增益的0.6%。这些结果将CASTER定位为一种适用于冻结模型部署的轻量级适应机制，同时明确说明了其安全信号在何种情况下具有信息量、在何种情况下不具有信息量。

English

Test-time adaptation (TTA) typically assumes that model parameters can be updated at inference time. This assumption is restrictive for inference-only accelerators, frozen or third-party models, and memory-constrained deployments, and standard BatchNorm-based TTA configurations may also become inactive on architectures without BatchNorm. We study adaptation when the learned model must remain frozen. We introduce CASTER, a gradient-free method that stores source class statistics in a discriminative subspace, estimates a class-shared affine transformation from target-batch moments, and analytically transports the source class distributions before classification. CASTER requires no backward pass, optimizer state, or stored source feature bank. Across four backbones and seven datasets, it outperforms k-NN on identical frozen features in 27 of 28 backbone-dataset settings while retaining a median of 18x less state. Affine transport is not always reliable. On ImageNet-C, where batches contain only 64 samples for 1000 classes, unconditional transport loses 21.2 top-1 points. We therefore introduce an empirical residual-to-margin transportability certificate. Across 307 evaluation cells, every transport losing more than 10 points has certificate value above 3.9, although benign and destructive regimes are not perfectly separated. Gating converts an average -3.35-point effect of unconditional transport into a +1.69-point gain, and performance remains within 0.3 points of the best threshold over a broad threshold range. Finally, we show that this certificate is mechanism-specific: when applied to Tent, it accepts only 4.3% of updates and preserves 0.6% of Tent's available gain. These results position CASTER as a lightweight adaptation mechanism for frozen-model deployment, together with an explicit account of when its safety signal is informative and when it is not.