3D Visual Illusion Depth Estimation

Yuwei Wu (Beijing Institute of Technology) · Yunde Jia (Shenzhen MSU-BIT University) · Chengtang Yao (Beijing Institute of Technology) · Zhidan Liu (Beijing Institute of Technology) · Jiaxi Zeng (Beijing Institute of Technology) · Lidong Yu (NVIDIA)

3d visual illusionadaptive fusionbinocular depth estimationbinocular disparitydataset collectiondepth estimation frameworkexperimental evaluationillusion impact analysismachine visual systemmonocular depth estimationmulti-view depth estimationperceptual phenomenonperformance metrics sota methods vision language model

3D visual illusion is a perceptual phenomenon where a two-dimensional plane is manipulated to simulate three-dimensional spatial relationships, making a flat artwork or object look three-dimensional in the human visual system. In this paper, we reveal that the machine visual system is also seriously fooled by 3D visual illusions, including monocular and binocular depth estimation. In order to explore and analyze the impact of 3D visual illusion on depth estimation, we collect a large dataset containing almost 3k scenes and 200k images to train and evaluate SOTA monocular and binocular depth estimation methods. We also propose a 3D visual illusion depth estimation framework that uses common sense from the vision language model to adaptively fuse depth from binocular disparity and monocular depth. Experiments show that SOTA monocular, binocular, and multi-view depth estimation approaches are all fooled by various 3D visual illusions, while our method achieves SOTA performance.