PandaPose: 3D Human Pose Lifting from a Single Image via Propagating 2D Pose Prior to 3D Anchor Space

Hang Xu (Huawei Noah’s Ark Lab) · Jiaqi Li (Beijing Institute for General Artificial Intelligence) · Joey Tianyi Zhou (CFAR, A*STAR) · Yang Xiao (Hong Kong Polytechnic University) · Jinghong Zheng (Huazhong University of Science and Technology) · Changlong Jiang (Huazhong University of Science and Technology) · Haohong Kuang (Huazhong University of Science and Technology) · Ran Wang (Huazhong University of Science and Technology) · Zhiguo Cao (Huazhong University of Science and Technology) · Min Du (ByteDance Inc.)
2d features3d anchor space3d human pose liftinganchor-feature interaction decoderdepth-aware feature liftingensemble predictionerror propagationhuman3.6mjoint-to-joint mappingjoint-wise 3d anchorspandaposergb imageself-occlusionsota methodsunified anchor queries

3D human pose lifting from a single RGB image is a challenging task in 3D vision. Existing methods typically establish a direct joint-to-joint mapping from 2D to 3D poses based on 2D features. This formulation suffers from two fundamental limitations: inevitable error propagation from input predicted 2D pose to 3D predictions and inherent difficulties in handling self-occlusion cases. In this paper, we propose PandaPose, a 3D human pose lifting approach via propagating 2D pose prior to 3D anchor space as the unified intermediate representation. Specifically, our 3D anchor space comprises: (1) Joint-wise 3D anchors in the canonical coordinate system, providing accurate and robust priors to mitigate 2D pose estimation inaccuracies. (2) Depth-aware joint-wise feature lifting that hierarchically integrates depth information to resolve self-occlusion ambiguities. (3) The anchor-feature interaction decoder that incorporates 3D anchors with lifted features to generate unified anchor queries encapsulating joint-wise 3D anchor set, visual cues and geometric depth information. The anchor queries are further employed to facilitate anchor-to-joint ensemble prediction. Experiments on three well-established benchmarks (i.e., Human3.6M, MPI-INF-3DHP and 3DPW) demonstrate the superiority of our proposition. The substantial reduction in error by 14.7% compared to SOTA methods on the challenging conditions of Human3.6M and qualitative comparisons further showcase the effectiveness and robustness of our approach.