HouseLayout3D: A Benchmark and Training-free Baseline for 3D Layout Estimation in the Wild

Marie-Julie Rakotosaona (Google) · Keisuke Tateno · Valentin Bieri (ETHZ - ETH Zurich) · Francis Engelmann (Computer Science Department, Stanford University) · Leonidas Guibas (stanford.edu)

2d segmentation3d layout estimation3d reconstructionarchitecturally complex spacesbaseline method benchmark dataset global contexthouselayout3dlayout-specific trainingmulti-floor buildingsmultifloor3dper-floor processingreal-world benchmarkstate-of-the-art methods synthetic datasets

Current 3D layout estimation models are predominantly trained on synthetic datasets biased toward simplistic, single-floor scenes. This prevents them from generalizing to complex, multi-floor buildings, often forcing a per-floor processing approach that sacrifices global context. Few works have attempted to holistically address multi-floor layouts. In this work, we introduce HouseLayout3D, a real-world benchmark dataset, which highlights the limitations of existing research when handling expansive, architecturally complex spaces. Additionally, we propose MultiFloor3D, a baseline method leveraging recent advances in 3D reconstruction and 2D segmentation. Our approach significantly outperforms state-of-the-art methods on both our new and existing datasets. Remarkably, it does not require any layout-specific training.