FlowR2A：學習多模態駕駛規劃的獎勵至動作分佈

摘要

多模態駕駛規劃長期以來面臨兩種範式之間的張力：基於評分的方法受益於密集的獎勵監督，但受限於固定的動作詞彙表；而基於錨點的方法雖能動態生成提議，卻因僅受限於單一真實軌跡而遭受稀疏監督之困。在本工作中，我們提出FlowR2A，通過將基於模擬的獎勵從判別式目標重構為生成式條件，從而化解此張力。藉由以流匹配解碼器從密集的軌跡-獎勵對中學習獎勵條件動作分佈，FlowR2A在單一生成模型中統一了基於評分方法的密集監督與基於錨點方法的提議生成，迫使模型內化動作與其在安全性、進展、舒適度及規則遵守方面結果之間的關聯。為平衡硬安全約束與軟進展目標，我們引入了細粒度的逐時間步獎勵條件化與獎勵噪聲增強。此生成式公式自然支持透過獎勵引導與錨點取樣進行受控的測試時取樣，從而產生高品質提議。FlowR2A在NAVSIM v1與v2基準中達到了最先進的成果，其多模態提議品質遠高於先前方法。

English

Multimodal driving planning faces a long-standing tension between two paradigms: scoring-based methods benefit from dense reward supervision but are confined to a fixed action vocabulary, while anchor-based methods generate proposals dynamically yet suffer from sparse supervision constrained to a single ground-truth trajectory. In this work, we propose FlowR2A, which resolves this tension by reframing simulation-based rewards from discriminative targets into generative conditions. By learning the reward-conditioned action distribution from dense trajectory-reward pairs with a flow-matching decoder, FlowR2A unifies the dense supervision of scoring-based methods with the proposal generation of anchor-based methods in a single generative model, forcing the model to internalize the correlation between an action and its outcomes in safety, progress, comfort, and rule compliance. To balance hard safety constraints against soft progress objectives, we introduce fine-grained per-timestep reward conditioning and reward noise augmentation. The generative formulation naturally supports controllable test-time sampling via reward guidance and anchored sampling, producing high-quality proposals. FlowR2A achieves state-of-the-art results on the NAVSIM v1 and v2 benchmarks, with multimodal proposals of substantially higher quality than prior methods.