SceneSplat++: A Large Dataset and Comprehensive Benchmark for Language Gaussian Splatting

Danda Pani Paudel (INSAIT Sofia & ETH Zurich) · Luc V Gool (Computer Vision Lab, ETH Zurich) · Nicu Sebe (University of Trento) · Yue Li (University of Amsterdam) · Ender Konukoglu (ETH Zurich) · Martin R. Oswald (University of Amsterdam) · Mengjiao Ma (INSAIT) · Qi Ma (ETH Zurich, INSAIT Soifa) · Jiahuan Cheng (Johns Hopkins University) · Runyi Yang (Institute for Computer Science, Artificial Intelligence and Technology) · Bin Ren (University of Pisa) · Nikola Popovic (INSAIT, Sofia University) · Mingqiang Wei (Nanjing University of Aeronautics and Astronautics) · Theo Gevers (University of Amsterdam 3DUniversum)
2d rendering3d gaussian splatting3d scene understandingdata priorsdiverse scenesfeed-forward inferencegeneralizable approachholistic 3d understandinglanguage groundinglarge-scale benchmarkoptimization-based methodsoptimization-free methodsscene geometryscenesplat-49ksegmentation performance

3D Gaussian Splatting (3DGS) serves as a highly performant and efficient encoding of scene geometry, appearance, and semantics. Moreover, grounding language in 3D scenes has proven to be an effective strategy for 3D scene understanding. Current Language Gaussian Splatting line of work fall into three main groups: (i) per-scene optimization-based, (ii) per-scene optimization-free, and (iii) generalizable approach. However, most of them are evaluated only on rendered 2D views of a handful of scenes and viewpoints close to the training views, limiting ability and insight into holistic 3D understanding. To address this gap, we propose the first large-scale benchmark that systematically assesses these three groups of methods directly in 3D space, evaluating on 1060 scenes across three indoor datasets and one outdoor dataset. Benchmark results demonstrate a clear advantage of the generalizable paradigm, particularly in relaxing the scene-specific limitation, enabling fast feed-forward inference on novel scenes, and achieving superior segmentation performance. We further introduce SceneSplat-49K -- a carefully curated 3DGS dataset comprising of around 49K diverse indoor and outdoor scenes trained from multiple sources, with which we demonstrate generalizable approach could harness strong data priors. Our codes, benchmark, and datasets are available.