Timezone: »

SNAKE: Shape-aware Neural 3D Keypoint Field
Chengliang Zhong · Peixing You · Xiaoxue Chen · Hao Zhao · Fuchun Sun · Guyue Zhou · Xiaodong Mu · Chuang Gan · Wenbing Huang


Detecting 3D keypoints from point clouds is important for shape reconstruction, while this work investigates the dual question: can shape reconstruction benefit 3D keypoint detection? Existing methods either seek salient features according to statistics of different orders or learn to predict keypoints that are invariant to transformation. Nevertheless, the idea of incorporating shape reconstruction into 3D keypoint detection is under-explored. We argue that this is restricted by former problem formulations. To this end, a novel unsupervised paradigm named SNAKE is proposed, which is short for shape-aware neural 3D keypoint field. Similar to recent coordinate-based radiance or distance field, our network takes 3D coordinates as inputs and predicts implicit shape indicators and keypoint saliency simultaneously, thus naturally entangling 3D keypoint detection and shape reconstruction. We achieve superior performance on various public benchmarks, including standalone object datasets ModelNet40, KeypointNet, SMPL meshes and scene-level datasets 3DMatch and Redwood. Intrinsic shape awareness brings several advantages as follows. (1) SNAKE generates 3D keypoints consistent with human semantic annotation, even without such supervision. (2) SNAKE outperforms counterparts in terms of repeatability, especially when the input point clouds are down-sampled. (3) the generated keypoints allow accurate geometric registration, notably in a zero-shot setting. Codes and models are available at https://github.com/zhongcl-thu/SNAKE.

Author Information

Chengliang Zhong (Tsinghua University)
Peixing You (Tsinghua University, Tsinghua University)
Xiaoxue Chen (Tsinghua University, Tsinghua University)
Hao Zhao (Tsinghua University, Tsinghua University)
Fuchun Sun (Tsinghua University)
Guyue Zhou (Tsinghua University)
Xiaodong Mu (Xi'an University of Electronic Science and Technology)
Chuang Gan (UMass Amherst/ MIT-IBM Watson AI Lab)
Wenbing Huang (Tsinghua University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors