Timezone: »

 
Poster
UViM: A Unified Modeling Approach for Vision with Learned Guiding Codes
Alexander Kolesnikov · André Susano Pinto · Lucas Beyer · Xiaohua Zhai · Jeremiah Harmsen · Neil Houlsby

Thu Dec 01 09:00 AM -- 11:00 AM (PST) @ Hall J #124
in Poster Session 5 »

We introduce UViM, a unified approach capable of modeling a wide range of computer vision tasks. In contrast to previous models, UViM has the same functional form for all tasks; it requires no task-specific modifications which require extensive human expertise. The approach involves two components: (I) a base model (feed-forward) which is trained to directly predict raw vision outputs, guided by a learned discrete code and (II) a language model (autoregressive) that is trained to generate the guiding code. These components complement each other: the language model is well-suited to modeling structured interdependent data, while the base model is efficient at dealing with high-dimensional outputs. We demonstrate the effectiveness of UViM on three diverse and challenging vision tasks: panoptic segmentation, depth prediction and image colorization, where we achieve competitive and near state-of-the-art results. Our experimental results suggest that UViM is a promising candidate for a unified modeling approach in computer vision.

Keywords: Deep Learning · Computer Vision · unified model · discrete representations

Author Information

Alexander Kolesnikov (Google Research, Brain team)
André Susano Pinto (Google)
Lucas Beyer (Google Brain Zürich)
Xiaohua Zhai (Google Brain)
Jeremiah Harmsen (Google Brain)
Neil Houlsby (Google)

More from the Same Authors