ProDyG: Progressive Dynamic Scene Reconstruction via Gaussian Splatting from Monocular Videos

Siyuan Li (Westlake University) · Shi Chen (ETHZ - ETH Zurich) · Erik Sandström (ETHZ - ETH Zurich) · Sandro Lombardi (Independent) · Martin R. Oswald (University of Amsterdam)

appearance modelingdynamic 3d reconstructiondynamic scene reconstructionglobal pose consistencymap consistencymotion masking strategymotion scaffolds graphnovel view renderingsonline operationprogressive adaptationrgb-d inputsslam methodsstate-of-the-art dynamic slamstatic and dynamic partstracking robustness

Achieving truly practical dynamic 3D reconstruction requires online operation, global pose and map consistency, detailed appearance modeling, and the flexibility to handle both RGB and RGB-D inputs. However, existing SLAM methods typically merely remove the dynamic parts or require RGB-D input, while offline methods are not scalable to long video sequences, and current transformer-based feedforward methods lack global consistency and appearance details. To this end, we achieve online dynamic scene reconstruction by disentangling the static and dynamic parts within a SLAM system. The poses are tracked robustly with a novel motion masking strategy, and dynamic parts are reconstructed leveraging a progressive adaptation of a Motion Scaffolds graph. Our method yields novel view renderings competitive to offline methods and achieves on-par tracking with state-of-the-art dynamic SLAM methods.