Surface-Aware Feed-Forward Quadratic Gaussian for Frame Interpolation with Large Motion

Zaoming Yan (East China Normal University) · Yaomin Huang (East China Normal University) · Pengcheng Lei (East China Normal University) · Qizhou Chen (East China Normal University) · Guixu Zhang (East China Normal University) · Faming Fang (East China Normal University)

2d frame space3d spacedifferential surface coordinate systemdifferential surface theoryexperimental findingsframe interpolationglobal receptive fieldsimage coordinate systemlarge-motion scenariosobject correspondencesstate-of-the-art performancesurface geometric propertiessurface uniqueness theoremvfi subsetsvideo frames transformation

Motion in the real world takes place in 3D space. Existing Frame Interpolation methods often estimate global receptive fields in 2D frame space. Due to the limitations of 2D space, these global receptive fields are limited, which makes it difficult to match object correspondences between frames, resulting in sub-optimal performance when handling large-motion scenarios. In this paper, we introduce a novel pipeline for exploring object correspondences based on differential surface theory. The differential surface coordinate system provides a better representation of the real world, enabling effective exploration of object correspondences. Specifically, the pipeline first transforms an input pair of video frames from the image coordinate system to the differential surface coordinate system. Subsequently, within this coordinate system, object correspondences are explored based on surface geometric properties and the surface uniqueness theorem. Experimental findings showcase that our method attains state-of-the-art performance across large motion benchmarks. Our method demonstrates the state-of-the-art performance on these VFI subsets with large motion.