A Unified Solution to Video Fusion: From Multi-Frame Learning to Benchmarking

Kai Zhang (The Ohio State University) · Yulun Zhang (Shanghai Jiao Tong University) · Konrad Schindler (ETH Zürich) · Zixiang Zhao (ETH Zurich) · Haowen Bai (Xi'an Jiaotong University) · Bingxin Ke (ETHZ - ETH Zurich) · Yukun Cui (Xi'an Jiaotong University) · Lilun Deng (Xi'an Jiaotong University)
comprehensive benchmarkevaluation protocolfeature warpinginfrared-visiblemedical fusionmulti-exposuremulti-focusmulti-frame learningoptical flowspatial qualitystate-of-the-art resultssynthetic data generationtemporal coherencetemporal correlationsvideo fusion

The real world is dynamic, yet most image fusion methods process static frames independently, ignoring temporal correlations in videos and leading to flickering and temporal inconsistency. To address this, we propose Unified Video Fusion (UniVF), a novel and unified framework for video fusion that leverages multi-frame learning and optical flow-based feature warping for informative, temporally coherent video fusion. To support its development, we also introduce Video Fusion Benchmark (VF-Bench), the first comprehensive benchmark covering four video fusion tasks: multi-exposure, multi-focus, infrared-visible, and medical fusion. VF-Bench provides high-quality, well-aligned video pairs obtained through synthetic data generation and rigorous curation from existing datasets, with a unified evaluation protocol that jointly assesses the spatial quality and temporal consistency of video fusion. Extensive experiments show that UniVF achieves state-of-the-art results across all tasks on VF-Bench. Project page: [vfbench.github.io](https://vfbench.github.io).