PRIX：从原始像素学习规划，实现端到端自动驾驶

摘要

尽管端到端自动驾驶模型展现出令人瞩目的成果，但其实际部署常受限于庞大的模型规模、对昂贵激光雷达传感器的依赖以及计算密集型的鸟瞰图（BEV）特征表示。这尤其限制了其在仅配备摄像头的大众市场车辆中的可扩展性。为应对这些挑战，我们提出了PRIX（Plan from Raw Pixels，从原始像素规划）。这一新颖且高效的端到端驾驶架构仅利用摄像头数据运行，无需显式的BEV表示，也无需激光雷达。PRIX结合视觉特征提取器与生成式规划头，直接从原始像素输入预测安全轨迹。架构的核心组件是上下文感知重校准变换器（CaRT），这是一个旨在有效增强多层次视觉特征以实现更稳健规划的新颖模块。通过全面实验，我们证明PRIX在NavSim和nuScenes基准测试中达到了最先进的性能，与更大规模、多模态扩散规划器相媲美，同时在推理速度和模型大小上显著更为高效，使其成为现实世界部署的实用解决方案。我们的工作已开源，代码将发布于https://maxiuw.github.io/prix。

English

While end-to-end autonomous driving models show promising results, their practical deployment is often hindered by large model sizes, a reliance on expensive LiDAR sensors and computationally intensive BEV feature representations. This limits their scalability, especially for mass-market vehicles equipped only with cameras. To address these challenges, we propose PRIX (Plan from Raw Pixels). Our novel and efficient end-to-end driving architecture operates using only camera data, without explicit BEV representation and forgoing the need for LiDAR. PRIX leverages a visual feature extractor coupled with a generative planning head to predict safe trajectories from raw pixel inputs directly. A core component of our architecture is the Context-aware Recalibration Transformer (CaRT), a novel module designed to effectively enhance multi-level visual features for more robust planning. We demonstrate through comprehensive experiments that PRIX achieves state-of-the-art performance on the NavSim and nuScenes benchmarks, matching the capabilities of larger, multimodal diffusion planners while being significantly more efficient in terms of inference speed and model size, making it a practical solution for real-world deployment. Our work is open-source and the code will be at https://maxiuw.github.io/prix.

PRIX：从原始像素学习规划，实现端到端自动驾驶

PRIX: Learning to Plan from Raw Pixels for End-to-End Autonomous Driving

摘要

Support