Timezone: »
Global context is crucial for 3D point cloud scene understanding tasks. In this work, we extended the contextual encoding layer that was originally designed for 2D tasks to 3D Point Cloud scenarios. The encoding layer learns a set of code words in the feature space of the 3D point cloud to characterize the global semantic context, and then based on these code words, the method learns a global contextual descriptor to reweight the featuremaps accordingly. Moreover, compared to 2D scenarios, data sparsity becomes a major issue in 3D point cloud scenarios, and the performance of contextual encoding quickly saturates when the number of code words increases. To mitigate this problem, we further proposed a group contextual encoding method, which divides the channel into groups and then performs encoding on group-divided feature vectors. This method facilitates learning of global context in grouped subspace for 3D point clouds. We evaluate the effectiveness and generalizability of our method on three widely-studied 3D point cloud tasks. Experimental results have shown that the proposed method outperformed the VoteNet remarkably with 3 mAP on the benchmark of SUN-RGBD, with the metrics of mAP@ 0.25, and a much greater margin of 6.57 mAP on ScanNet with the metrics of mAP@ 0.5. Compared to the baseline of PointNet++, the proposed method leads to an accuracy of 86 %, outperforming the baseline by 1.5 %. Our proposed method have outperformed the non-grouping baseline methods across the board and establishes new state-of-the-art on these benchmarks.
Author Information
Xu Liu (The University of Tokyo)
Chengtao Li (MIT)
Jian Wang (Carnegie Mellon University)
Jingbo Wang (Peking University)
Boxin Shi (Peking University)
Xiaodong He (JD AI research)
More from the Same Authors
-
2022 Poster: Compressible-composable NeRF via Rank-residual Decomposition »
Jiaxiang Tang · Xiaokang Chen · Jingbo Wang · Gang Zeng -
2023 Poster: Slow and Weak Attractor Computation Embedded in Fast and Strong E-I Balanced Neural Dynamics »
Xiaohan Lin · Liyuan Li · Boxin Shi · Tiejun Huang · Yuanyuan Mi · Si Wu -
2023 Poster: L-CAD: Language-based Colorization with Any-level Descriptions »
zheng chang · Shuchen Weng · Peixuan Zhang · Yu Li · Si Li · Boxin Shi -
2023 Poster: LuminAIRe: Illumination-Aware Conditional Image Repainting for Lighting-Realistic Generation »
Jiajun Tang · Haofeng Zhong · Shuchen Weng · Boxin Shi -
2022 Spotlight: Neural Transmitted Radiance Fields »
Chengxuan Zhu · Renjie Wan · Boxin Shi -
2022 Spotlight: Compressible-composable NeRF via Rank-residual Decomposition »
Jiaxiang Tang · Xiaokang Chen · Jingbo Wang · Gang Zeng -
2022 Poster: Neural Transmitted Radiance Fields »
Chengxuan Zhu · Renjie Wan · Boxin Shi -
2021 Poster: Learning to dehaze with polarization »
Chu Zhou · Minggui Teng · Yufei Han · Chao Xu · Boxin Shi -
2020 Poster: UnModNet: Learning to Unwrap a Modulo Image for High Dynamic Range Imaging »
Chu Zhou · Hang Zhao · Jin Han · Chang Xu · Chao Xu · Tiejun Huang · Boxin Shi -
2020 Poster: GPS-Net: Graph-based Photometric Stereo Network »
Zhuokun Yao · Kun Li · Ying Fu · Haofeng Hu · Boxin Shi -
2019 Poster: Reflection Separation using a Pair of Unpolarized and Polarized Images »
Youwei Lyu · Zhaopeng Cui · Si Li · Marc Pollefeys · Boxin Shi -
2019 Spotlight: Reflection Separation using a Pair of Unpolarized and Polarized Images »
Youwei Lyu · Zhaopeng Cui · Si Li · Marc Pollefeys · Boxin Shi -
2019 Poster: Learning from Bad Data via Generation »
Tianyu Guo · Chang Xu · Boxin Shi · Chao Xu · Dacheng Tao -
2019 Poster: Aligning Visual Regions and Textual Concepts for Semantic-Grounded Image Representations »
Fenglin Liu · Yuanxin Liu · Xuancheng Ren · Xiaodong He · Xu Sun