Raw2Drive: Reinforcement Learning with Aligned World Models for End-to-End Autonomous Driving (in CARLA v2)

Zhenjie Yang (Shanghai Jiao Tong University) · Xiaosong Jia (University of California, Berkeley) · Qifeng Li (Shanghai Jiaotong University) · Xue Yang (Shanghai AI Laboratory) · Maoqing Yao (Agibot) · Junchi Yan (Shanghai Jiao Tong University)

auxiliary world modelcarla leaderboarddual-stream approachend-to-end autonomous drivingguidance mechanismimitation learning model-based reinforcement learningneural planningprior knowledgeprivileged informationraw sensor policyraw sensor world modelreinforcement learning rollouts state-of-the-art performancetraining consistency

Reinforcement Learning (RL) can mitigate the causal confusion and distribution shift inherent to imitation learning (IL). However, applying RL to end-to-end autonomous driving (E2E-AD) remains an open problem for its training difficulty, and IL is still the mainstream paradigm in both academia and industry. Recently Model-based Reinforcement Learning (MBRL) have demonstrated promising results in neural planning; however, these methods typically require privileged information as input rather than raw sensor data. We fill this gap by designing Raw2Drive, a dual-stream MBRL approach. Initially, we efficiently train an auxiliary privileged world model paired with a neural planner that uses privileged information as input. Subsequently, we introduce a raw sensor world model trained via our proposed Guidance Mechanism, which ensures consistency between the raw sensor world model and the privileged world model during rollouts. Finally, the raw sensor world model combines the prior knowledge embedded in the heads of the privileged world model to effectively guide the training of the raw sensor policy. Raw2Drive is so far the only RL based end-to-end method on CARLA Leaderboard 2.0, and Bench2Drive and it achieves state-of-the-art performance.