Reasoning Beyond Points: A Visual Introspective Approach for Few-Shot 3D Segmentation

Yixian Shen (University of Amsterdam) · Shuting He (Shanghai University of Finance and Economics) · Xiang Fang (Huazhong University of Science and Technology) · Changshuo Wang (University College London, University of London) · Zhijian Hu (LAAS / CNRS) · JIA-HONG HUANG (University of Amsterdam) · Prayag Tiwari (Halmstad University)

detailed featuresdomain gapsdynamic power convolutionintra-class diversitylearnable power functionslocal geometric structuresmulti-step reasoningpoint cloud few-shot semantic segmentationprototype difference moduleprototype discriminabilityprototype enhancement modules3disscannetsemantic inconsistenciesvisual introspective prototype segmentation

Point Cloud Few-Shot Semantic Segmentation (PC-FSS) aims to segment unknown categories in query samples using only a small number of annotated support samples. However, scene complexity and insufficient representation of local geometric structures pose significant challenges to PC-FSS. To address these issues, we propose a novel pre-training-free Visual Introspective Prototype Segmentation network (VIP-Seg). Specifically, we design a Visual Introspective Prototype (VIP) module that employs a multi-step reasoning approach to tackle intra-class diversity and domain gaps between support and query sets. The VIP module consists of a Prototype Enhancement Module (PEM) and a Prototype Difference Module (PDM), which work alternately to progressively refine prototypes. The PEM enhances prototype discriminability and reduces intra-class diversity, while the PDM learns common representations from the differences between query and support features, effectively eliminating semantic inconsistencies caused by domain gaps. To further reduce intra-class diversity and enhance point discriminative ability, we propose a Dynamic Power Convolution (DyPowerConv) that leverages learnable power functions to effectively capture local geometric structures and detailed features of point clouds. Extensive experiments on S3DIS and ScanNet demonstrate that our proposed VIP-Seg significantly outperforms current state-of-the-art methods, proving its effectiveness in PC-FSS tasks. Our code will be available at https://github.com/changshuowang/VIP-Seg.