IBGS: Image-Based Gaussian Splatting

Miaomiao Liu (Australian National University) · Hoang Chuong Nguyen (Australia National University) · Wei Mao (Meta) · Jose M. Alvarez (NVIDIA)

3d gaussian splattingglobal texture maphigh-frequency detailshigh-resolution source imagesimage-based gaussian splattinglearned residualnovel view synthesisper-gaussian texture mapsrendering qualityspatially varying colorspecular highlightsspherical harmonicssurface alignmentview-dependent effectsview-specific color modeling

3D Gaussian Splatting (3DGS) has recently emerged as a fast, high-quality method for novel view synthesis (NVS). However, its use of low-degree spherical harmonics limits its ability to capture spatially varying color and view-dependent effects such as specular highlights. Existing works augment Gaussians with either a global texture map, which struggles with complex scenes, or per-Gaussian texture maps, which introduces high storage overhead. We propose Image-Based Gaussian Splatting, an efficient alternative that leverages high-resolution source images for fine details and view-specific color modeling. Specifically, we model each pixel color as a combination of a base color from standard 3DGS rendering and a learned residual inferred from neighboring training images. This promotes accurate surface alignment and enables rendering images of high-frequency details and accurate view-dependent effects. Experiments on standard NVS benchmarks show that our method significantly outperforms prior Gaussian Splatting approaches in rendering quality, without increasing the storage footprint.