DrivingRecon: Large 4D Gaussian Reconstruction Model For Autonomous Driving

Wenzhao Zheng (University of California, Berkeley) · Yingcong Chen (The Chinese University of Hong Kong) · Masayoshi TOMIZUKA (MSC Lab) · Kurt Keutzer (EECS, UC Berkeley) · Hao LU (Hong Kong University of Science and Technology) · Tianshuo Xu · Yunpeng Zhang (Tsinghua University) · Wei Zhan (University of California, Berkeley) · Dalong Du (PhiGent Robotics)

3d representations4d representationsautonomous drivingdrivingrecondynamic decouplinggaussian pointslarge 4d gaussian reconstruction modellarge reconstruction model model pre-trainingpd-blockprune and dilate blockscene reconstruction qualitystatic decouplingtemporary-consistent geometryvehicle type adaptation

Large reconstruction model has remarkable progress, which can directly predict 3D or 4D representations for unseen scenes and objects. However, current work has not systematically explored the potential of large reconstruction models in the field of autonomous driving. To achieve this, we introduce the Large 4D Gaussian Reconstruction Model (DrivingRecon). With an elaborate and simple framework design, it not only ensures efficient and high-quality reconstruction, but also provides potential for downstream tasks. There are two core contributions: firstly, the Prune and Dilate Block (PD-Block) is proposed to prune redundant and overlapping Gaussian points and dilate Gaussian points for complex objects. Then, dynamic and static decoupling is tailored to better learn the temporary-consistent geometry across different time. Experimental results demonstrate that DrivingRecon significantly improves scene reconstruction quality compared to existing methods. Furthermore, we explore applications of DrivingRecon in model pre-training, vehicle type adaptation, and scene editing. Our code will be available.