SEGA: Shaping Semantic Geometry for Robust Hashing under Noisy Supervision

Ming Zhang (North China Electric Power University) · Dacheng Tao (Nanyang Technological University) · Xiao Luo (UCLA) · Wei Ju (Sichuan University) · Yiyang Gu (Peking University) · Bohan Wu (Peking University) · Qinghua Ran (Peking University) · Rong-Cheng Tu (Beijing Institute of Technology) · Zhiping Xiao (University of Washington, Computer Science & Engineering)

class boundariesdynamic class prototypesgeometric structurehash codeshash embeddingshash spacelabel denoisingnoisy supervisionpredicted distributionssemantic anchorssemantic geometry shapingsoft calibrationstructural stabilitystructure-aware interpolationstructure-based divergenceuncertainty estimation

This paper studies the problem of learning hash codes from noisy supervision, which is a practical yet challenging task. This problem is important in extensive real-world applications such as image retrieval and cross-modal retrieval. However, most of the existing methods focus on label denoising to address this problem, but ignore the geometric structure of the hash space, which is critical for learning stable hash codes. Towards this end, this paper proposes a novel framework named Semantic Geometry Shaping (SEGA) that explicitly refines the semantic geometry of hash space. Specifically, we first learn dynamic class prototypes as semantic anchors and cluster hash embeddings around these prototypes to keep structural stability. We then leverage both the energy of predicted distributions and structure-based divergence to estimate the uncertainty of instances and calibrate the supervision in a soft manner. Moreover, we introduce structure-aware interpolation to improve the class boundaries. To verify the effectiveness of our design, we give the theoretical analysis for the proposed framework. Experiments on a range of widely-used retrieval datasets justify the superiority of our SEGA over extensive strong baselines under noisy supervision.