RaysUp: 通过几何感知射线表示的超轻量通用特征上采样

摘要

预训练视觉基础模型（VFMs）凭借其强大的语义表示能力和泛化性能，已成为现代计算机视觉的核心支柱。然而，其 patch 化或池化输出本质上是低分辨率的，这限制了它们在需要细粒度像素级推理的任务中的有效性。现有特征上采样方法要么降低语义保真度，要么依赖 VFM 特定的重新训练和重型架构，导致效率和可扩展性不足。为应对这些挑战，我们提出 RaysUp——一种超轻量级、任务无关且 VFM 无关的特征上采样框架，能够在任意分辨率下重建高分辨率特征图。与传统二维插值或注意力机制不同，RaysUp 将特征重建提升至几何感知的光线域。具体而言，我们引入用于方向感知引导编码的空间解耦引导编码器、用于分辨率灵活重建的任意分辨率交叉注意力机制，以及一种新颖的光线位置编码（RayPE），它通过 6D 普吕克光线坐标注入隐式三维几何先验。最后，几何感知邻域注意力模块进一步确保内容自适应的双边聚合，同时保持几何一致性。在多种密集预测任务上的大量实验表明，RaysUp 实现了最先进的性能，同时参数数量仅为 AnyUp 的 16%，推理速度提升约 7 倍。这些结果凸显了其显著改善的精度-效率权衡，并使 RaysUp 成为通用特征上采样的实用且可扩展的解决方案。代码地址：https://github.com/MAP-RaysUp/RaysUp。

English

Pre-trained Vision Foundation Models (VFMs) have become central to modern computer vision due to their powerful semantic representations and strong generalization ability. However, their patchified or pooled outputs are inherently low-resolution, limiting their effectiveness in tasks requiring fine-grained, pixel-level reasoning. Existing feature upsampling approaches either degrade semantic fidelity or rely on VFM-specific retraining and heavy architectures, hindering efficiency and scalability. To address these challenges, we propose RaysUp, an ultra-lightweight, task-agnostic, and VFM-agnostic feature upsampling framework that reconstructs high-resolution feature maps at arbitrary resolutions. Unlike conventional 2D interpolation or attention-based schemes, RaysUp lifts feature reconstruction into a geometry-aware ray domain. Specifically, we introduce a Spatially Decoupled Guidance Encoder for direction-aware guidance encoding, an Any-Resolution Cross-Attention mechanism for resolution-flexible reconstruction, and a novel Ray Positional Encoding (RayPE) that injects implicit 3D geometric priors via 6D Plucker ray coordinates. Finally, a Geometry-Aware Neighborhood Attention module further ensures content-adaptive bilateral aggregation while preserving geometric consistency. Extensive experiments across diverse dense prediction tasks demonstrate that RaysUp achieves state-of-the-art performance while using only 16% of the parameters of AnyUp and delivering approximately 7x faster inference. These results highlight a substantially improved accuracy-efficiency trade-off and establish RaysUp as a practical and scalable solution for universal feature upsampling. Code is available at https://github.com/MAP-RaysUp/RaysUp.