Towards Generalizable Retina Vessel Segmentation with Deformable Graph Priors

Ke Liu (Zhejiang University) · Shangde Gao (Zhejiang University) · Yichao Fu (University of California, San Diego) · Shangqi Gao (University of Cambridge)

anatomical graph priorsanatomical topologycross-domain segmentationdeformable graph priordifferentiable alignmentdomain shiftsdomain-invariant representationretinal vessel segmentationrobust generalizable segmentationstatistical retinal atlasstructure-aware image decompositionstructure-degraded componentsstructure-preserved componentsunsupervised energy functionvariational bayesian framework

Retinal vessel segmentation is critical for medical diagnosis, yet existing models often struggle to generalize across domains due to appearance variability, limited annotations, and complex vascular morphology. We propose GraphSeg, a variational Bayesian framework that integrates anatomical graph priors with structure-aware image decomposition to enhance cross-domain segmentation. GraphSeg factorizes retinal images into structure-preserved and structure-degraded components, enabling domain-invariant representation. A deformable graph prior, derived from a statistical retinal atlas, is incorporated via a differentiable alignment and guided by an unsupervised energy function. Experiments on three public benchmarks (CHASE, DRIVE, HRF) show that GraphSeg consistently outperforms existing methods under domain shifts. These results highlight the importance of jointly modeling anatomical topology and image structure for robust generalizable vessel segmentation.