InternScenes: A Large-scale Simulatable Indoor Scene Dataset with Realistic Layouts

Bo Dai (Google DeepMind & Georgia Tech) · Hanqing Wang (Shanghai Artificial Intelligence Laboratory) · Tai WANG (The Chinese University of Hong Kong) · Jiangmiao Pang (Shanghai AI Laboratory ) · Luo Li (Shanghai Artificial Intelligence Laboratory) · Xudong XU (Shanghai AI Laboratory) · Zhaoyang Lyu (Shanghai AI Laboratory) · Weipeng Zhong (Shanghai Jiao Tong University) · Peizhou Cao (Beijing University of Aeronautics and Astronautics) · Yichen Jin (Southeast University) · Wenzhe Cai (Shanghai Artificial Intelligence Laboratory) · Jingli Lin (Shanghai Jiaotong University)

complex scenesdata scaledesigner-created scenesembodied aiinteractive objectsinternscenesmodel trainingobject collisionsphysical simulationspoint-goal navigationprocedurally generated scenesreal-world scansrealistic layoutsscene diversityscene layout generationsimulatable 3d scene datasets

The advancement of Embodied AI heavily relies on large-scale, simulatable 3D scene datasets characterized by scene diversity and realistic layouts.However, existing datasets typically suffer from limitations in data scale or diversity, sanitized layouts lacking small items, and severe object collisions.To address these shortcomings, we introduce \textbf{InternScenes}, a novel large-scale simulatable indoor scene dataset comprising approximately 40,000 diverse scenes by integrating three disparate scene sources, \ie, real-world scans, procedurally generated scenes, and designer-created scenes, including 1.96M 3D objects and covering 15 common scene types and 288 object classes.We particularly preserve massive small items in the scenes, resulting in realistic and complex layouts with an average of 41.5 objects per region.Our comprehensive data processing pipeline ensures simulatability by creating real-to-sim replicas for real-world scans, enhances interactivity by incorporating interactive objects into these scenes, and resolves object collisions by physical simulations.We demonstrate the value of InternScenes with two benchmark applications: scene layout generation and point-goal navigation. Both show the new challenges posed by the complex and realistic layouts. More importantly, InternScenes paves the way for scaling up the model training for both tasks, making the generation and navigation in such complex scenes possible. We commit to open-sourcing the data and benchmarks to benefit the whole community.