SRSR: Enhancing Semantic Accuracy in Real-World Image Super-Resolution with Spatially Re-Focused Text-Conditioning

Violetta Shevchenko (Pluralis Research) · Chen Chen (University of Central Florida) · Chang Xu (University of Sydney) · Majid Abdolshah (Amazon) · Hongdong Li (Australian National University) · Pulak Purkait (Amazon)

cross-attentiondiffusion-based super-resolutionfidelity metricshallucinated detailsperceptual quality measurespsnrsemantic ambiguitiessemantic misalignmentspatially re-focused cross-attentionspatially re-focused super-resolutionspatially targeted classifier-free guidancessimtext conditioningungrounded pixelsvisually-grounded segmentation masks

Existing diffusion-based super-resolution approaches often exhibit semantic ambiguities due to inaccuracies and incompleteness in their text conditioning, coupled with the inherent tendency for cross-attention to divert towards irrelevant pixels. These limitations can lead to semantic misalignment and hallucinated details in the generated high-resolution outputs. To address these, we propose a novel, plug-and-play *spatially re-focused super-resolution (SRSR)* framework that consists of two core components: first, we introduce Spatially Re-focused Cross-Attention (SRCA), which refines text conditioning at inference time by applying visually-grounded segmentation masks to guide cross-attention. Second, we introduce a Spatially Targeted Classifier-Free Guidance (STCFG) mechanism that selectively bypasses text influences on ungrounded pixels to prevent hallucinations. Extensive experiments on both synthetic and real-world datasets demonstrate that SRSR consistently outperforms seven state-of-the-art baselines in standard fidelity metrics (PSNR and SSIM) across all datasets, and in perceptual quality measures (LPIPS and DISTS) on two real-world benchmarks, underscoring its effectiveness in achieving both high semantic fidelity and perceptual quality in super-resolution.