無梯度適應：仿射統計傳輸及其證書所能傳達的資訊

摘要

測試時適應（TTA）通常假設模型參數可在推論期間更新。此一假設對僅供推論的加速器、凍結或第三方模型，以及記憶體受限的部署而言具限制性；此外，標準的以BatchNorm為基礎之TTA設定，在沒有BatchNorm的架構上也可能失效。我們研究在學習模型必須保持凍結時的適應問題。我們提出CASTER，一種無梯度方法：在判別性子空間中儲存來源類別統計量，並由目標批次的動差估計一個類別共用的仿射變換，在分類前以解析方式將來源類別分佈傳輸至目標。CASTER不需要反向傳播、優化器狀態，也不需要儲存的來源特徵庫。在四個骨幹網路與七個資料集上，於28個骨幹-資料集設定中，有27個設定表現優於使用相同凍結特徵的k-NN，且其中位數狀態量僅為k-NN的1/18。仿射傳輸並非總是可靠。在ImageNet-C上，每個批次僅含64個樣本卻涵蓋1000個類別，無條件傳輸損失了21.2個top-1百分點。因此，我們引入一個以殘差-邊際比為基礎的經驗性可傳輸性驗證。在307個評估單元中，所有損失超過10個百分點的傳輸，其驗證值皆高於3.9；不過，良性與破壞性情形的分佈並非完全分離。門控機制能將無條件傳輸平均-3.35個百分點的效果轉為+1.69個百分點的增益，且在一段廣闊的閾值範圍內，效能與最佳閾值之間的差距維持在0.3個百分點以內。最後，我們證明此驗證具有機制特異性：將其應用於Tent時，僅接受4.3%的更新，並保留了Tent可用增益的0.6%。這些結果將CASTER定位為適用於凍結模型部署的輕量級適應機制，同時清楚說明其安全訊號何時具有資訊性、何時不具有。

English

Test-time adaptation (TTA) typically assumes that model parameters can be updated at inference time. This assumption is restrictive for inference-only accelerators, frozen or third-party models, and memory-constrained deployments, and standard BatchNorm-based TTA configurations may also become inactive on architectures without BatchNorm. We study adaptation when the learned model must remain frozen. We introduce CASTER, a gradient-free method that stores source class statistics in a discriminative subspace, estimates a class-shared affine transformation from target-batch moments, and analytically transports the source class distributions before classification. CASTER requires no backward pass, optimizer state, or stored source feature bank. Across four backbones and seven datasets, it outperforms k-NN on identical frozen features in 27 of 28 backbone-dataset settings while retaining a median of 18x less state. Affine transport is not always reliable. On ImageNet-C, where batches contain only 64 samples for 1000 classes, unconditional transport loses 21.2 top-1 points. We therefore introduce an empirical residual-to-margin transportability certificate. Across 307 evaluation cells, every transport losing more than 10 points has certificate value above 3.9, although benign and destructive regimes are not perfectly separated. Gating converts an average -3.35-point effect of unconditional transport into a +1.69-point gain, and performance remains within 0.3 points of the best threshold over a broad threshold range. Finally, we show that this certificate is mechanism-specific: when applied to Tent, it accepts only 4.3% of updates and preserves 0.6% of Tent's available gain. These results position CASTER as a lightweight adaptation mechanism for frozen-model deployment, together with an explicit account of when its safety signal is informative and when it is not.