Timezone: »
Semantic, instance, and panoptic segmentations have been addressed using different and specialized frameworks despite their underlying connections. This paper presents a unified, simple, and effective framework for these essentially similar tasks. The framework, named K-Net, segments both instances and semantic categories consistently by a group of learnable kernels, where each kernel is responsible for generating a mask for either a potential instance or a stuff class. To remedy the difficulties of distinguishing various instances, we propose a kernel update strategy that enables each kernel dynamic and conditional on its meaningful group in the input image. K-Net can be trained in an end-to-end manner with bipartite matching, and its training and inference are naturally NMS-free and box-free. Without bells and whistles, K-Net surpasses all previous published state-of-the-art single-model results of panoptic segmentation on MS COCO test-dev split and semantic segmentation on ADE20K val split with 55.2% PQ and 54.3% mIoU, respectively. Its instance segmentation performance is also on par with Cascade Mask R-CNN on MS COCO with 60%-90% faster inference speeds. Code and models will be released at https://github.com/ZwwWayne/K-Net/.
Author Information
Wenwei Zhang (Nanyang Technological University)
Jiangmiao Pang (Zhejiang University)
Kai Chen (The Chinese University of Hong Kong)
Chen Change Loy (Nanyang Technological University)
More from the Same Authors
-
2021 : Semi-Supervised Domain Generalization with Stochastic StyleMatch »
Kaiyang Zhou · Chen Change Loy · Ziwei Liu -
2021 Poster: A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis »
Xingang Pan · Xudong XU · Chen Change Loy · Christian Theobalt · Bo Dai -
2021 Poster: Few-Shot Object Detection via Association and DIscrimination »
Yuhang Cao · Jiaqi Wang · Ying Jin · Tong Wu · Kai Chen · Ziwei Liu · Dahua Lin -
2021 Poster: Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data »
Liming Jiang · Bo Dai · Wayne Wu · Chen Change Loy -
2021 Poster: Unsupervised Object-Level Representation Learning from Scene Images »
Jiahao Xie · Xiaohang Zhan · Ziwei Liu · Yew Soon Ong · Chen Change Loy -
2018 Poster: FishNet: A Versatile Backbone for Image, Region, and Pixel Level Prediction »
Shuyang Sun · Jiangmiao Pang · Jianping Shi · Shuai Yi · Wanli Ouyang -
2016 Poster: Local Similarity-Aware Deep Feature Embedding »
Chen Huang · Chen Change Loy · Xiaoou Tang