RGB生成から高密度場読み出しへ：テキスト画像モデルによるピクセル空間の密な予測

要旨

大规模文生图模型因其RGB生成预训练学习了丰富的语义、结构与几何先验，成为密集预测任务极具吸引力的骨干网络。现有生成与编辑方法将密集预测转化为目标生成任务以复用这些先验：将深度、法线、α遮罩、分割掩码及热力图等标注信息编码至经RGB训练的VAE潜在空间，再解码为类图像目标。我们认为这继承了比密集预测所需更多的生成式输出接口——与RGB合成不同，密集预测要求在同一图像平面上获得像素级正确的任务原生场，而非渲染新的RGB内容。我们的关键发现是：预训练DiT已通过图像平面上的“补丁→令牌→补丁”格点结构组织RGB输入，因此每个令牌索引一个固定输出补丁，其通道可承载任务原生量而非RGB外观。我们将其实现为ReChannel：保留用于DiT输入分布的VAE编码器，但丢弃目标侧解码器；通过任务LoRA适配冻结的DiT；并通过共享的令牌局部线性头（约33K参数，无空间混合）将每个令牌映射至其对应的p×p×K_t像素空间补丁。基于FLUX-Klein，我们在六项密集预测任务及十余个基准上进行了评估。这一最小化接口在无需三元图的抠图、KITTI深度估计及指代分割上取得了新的最优结果，并在法线、显著性及姿态估计上保持竞争力。在匹配的4B参数设定下，其精度更高，速度比编辑+潜在解码的对应方法快2.48倍——密集感知可从生成预训练中获益，而无需继承其输出接口。

English

Large-scale text-to-image models are attractive backbones for dense prediction because RGB generation pretraining learns rich semantic, structural, and geometric priors. Existing generative and editing approaches reuse these priors by casting dense prediction as target generation: annotations such as depth, normals, alpha mattes, masks, and heatmaps are encoded into an RGB-trained VAE latent space and decoded back as image-like targets. We argue this inherits more of the generative output interface than dense prediction requires: unlike RGB synthesis, dense prediction asks for pixel-correct, task-native fields on the same image plane, not new RGB content to be rendered. Our key observation is that a pretrained DiT already organizes RGB inputs through a patch-to-token-to-patch lattice on the image plane, so each token indexes a fixed output patch whose channels can carry task-native quantities instead of RGB appearance. We instantiate this as ReChannel: we keep the VAE encoder for the DiT's input distribution but drop the target-side decoder, adapt the frozen DiT with task LoRA, and map each token to its p x p x K_t pixel-space patch through a shared token-local linear head--about 33K parameters, no spatial mixing. Using FLUX-Klein, we evaluate on six dense prediction tasks and over a dozen benchmarks. This minimal interface sets new state-of-the-art on trimap-free matting, KITTI depth, and referring segmentation, and stays competitive on normals, saliency, and pose. In a matched 4B setting it is more accurate and 2.48x faster than an edit-plus-latent-decode counterpart--dense perception can benefit from generative pretraining without inheriting its output interface.