PolypSense3D: A Multi-Source Benchmark Dataset for Depth-Aware Polyp Size Measurement in Endoscopy

Lingjuan Lyu (Sony AI) · Ruyu Liu (Technical University of Denmark) · Lin Wang (Hangzhou Normal University) · Zhou Mingming (Hangzhou Normal University) · Jianhua Zhang (Tianjin University of Technology) · ZHANG HAOYU (Hangzhou Normal University) · Xiufeng Liu (Technical University of Denmark) · Xu Cheng (Tianjin University of Technology) · Sixian Chan (Zhejiang University of Technology) · Shen yanbin (Zhejiang University) · Dai Sheng (Zhejiang University) · Yuping Yan · Yaochu Jin (Westlake University)

automated pipelinescancer risk assessmentclinical sequencesdense/sparse depthdepth-aware measurementerror propagationforceps-assisted annotationmean absolute errormulti-source benchmark datasetphysical phantomspolyp sizingquantitative endoscopic visionsegmentation maskssynchronized rgbvirtual simulations

Accurate polyp sizing during endoscopy is crucial for cancer risk assessment but is hindered by subjective methods and inadequate datasets lacking integrated 2D appearance, 3D structure, and real-world size information. We introduce PolypSense3D, the first multi-source benchmark dataset specifically targeting depth-aware polyp size measurement. It uniquely integrates over 43,000 frames from virtual simulations, physical phantoms, and clinical sequences, providing synchronized RGB, dense/sparse depth, segmentation masks, camera parameters, and millimeter-scale size labels derived via a novel forceps-assisted in-vivo annotation technique. To establish its value, we benchmark state-of-the-art segmentation and depth estimation models. Results quantify significant domain gaps between simulated/phantom and clinical data and reveal substantial error propagation from perception stages to final size estimation, with the best fully automated pipelines achieving an average Mean Absolute Error (MAE) of 0.95 mm on the clinical data subset. Publicly released under CC BY-SA 4.0 with code and evaluation protocols, PolypSense3D offers a standardized platform to accelerate research in robust, clinically relevant quantitative endoscopic vision. The benchmark dataset and code are available at: https://github.com/HNUicda/PolypSense3D and https://doi.org/10.7910/DVN/K13H89.