RaysUp：基於幾何感知光線表示的超輕量通用特徵上採樣

摘要

預訓練視覺基礎模型（Vision Foundation Models, VFMs）因其強大的語義表示能力與優異的泛化性能，已成為現代電腦視覺的核心技術。然而，此類模型的塊狀化或池化輸出本質上解析度偏低，限制了其在需要細粒度、像素級推理任務中的效能。現有的特徵上取樣方法若非犧牲語義保真度，便是依賴於特定的VFM重新訓練與繁重架構，導致效率與可擴展性受阻。為解決這些問題，我們提出RaysUp——一個超輕量、任務無關且VFM無關的特徵上取樣框架，能在任意解析度下重建高解析度特徵圖。有別於傳統的二維插值或注意力機制方案，RaysUp將特徵重建提升至具幾何感知的光線域。具體而言，我們引入了空間解耦引導編碼器（Spatially Decoupled Guidance Encoder）以實現方向感知的引導編碼、任意解析度交叉注意力（Any-Resolution Cross-Attention）機制以達成解析度靈活的重建，以及新穎的光線位置編碼（Ray Positional Encoding, RayPE），透過六維普呂克光線座標注入隱式的三維幾何先驗。最後，幾何感知鄰域注意力模組（Geometry-Aware Neighborhood Attention）進一步確保內容自適應的雙邊聚合，同時維持幾何一致性。在多种密集預測任務上的大量實驗表明，RaysUp在僅使用AnyUp 16%參數量、且推理速度約快7倍的條件下，達到了最先進的性能。這些結果凸顯了大幅改善的精度-效率權衡，並確立了RaysUp作為通用特徵上取樣之實用且可擴展的解決方案地位。程式碼開源於 https://github.com/MAP-RaysUp/RaysUp。

English

Pre-trained Vision Foundation Models (VFMs) have become central to modern computer vision due to their powerful semantic representations and strong generalization ability. However, their patchified or pooled outputs are inherently low-resolution, limiting their effectiveness in tasks requiring fine-grained, pixel-level reasoning. Existing feature upsampling approaches either degrade semantic fidelity or rely on VFM-specific retraining and heavy architectures, hindering efficiency and scalability. To address these challenges, we propose RaysUp, an ultra-lightweight, task-agnostic, and VFM-agnostic feature upsampling framework that reconstructs high-resolution feature maps at arbitrary resolutions. Unlike conventional 2D interpolation or attention-based schemes, RaysUp lifts feature reconstruction into a geometry-aware ray domain. Specifically, we introduce a Spatially Decoupled Guidance Encoder for direction-aware guidance encoding, an Any-Resolution Cross-Attention mechanism for resolution-flexible reconstruction, and a novel Ray Positional Encoding (RayPE) that injects implicit 3D geometric priors via 6D Plucker ray coordinates. Finally, a Geometry-Aware Neighborhood Attention module further ensures content-adaptive bilateral aggregation while preserving geometric consistency. Extensive experiments across diverse dense prediction tasks demonstrate that RaysUp achieves state-of-the-art performance while using only 16% of the parameters of AnyUp and delivering approximately 7x faster inference. These results highlight a substantially improved accuracy-efficiency trade-off and establish RaysUp as a practical and scalable solution for universal feature upsampling. Code is available at https://github.com/MAP-RaysUp/RaysUp.