Poster
Positional Normalization
Boyi Li · Felix Wu · Kilian Weinberger · Serge Belongie
East Exhibition Hall B + C #147
Keywords: [ Deep Learning ] [ Applications -> Computer Vision; Deep Learning -> CNN Architectures; Deep Learning -> Deep Autoencoders; Deep Learning ] [ Gener ]
A widely deployed method for reducing the training time of deep neural networks is to normalize activations at each layer. Although various normalization schemes have been proposed, they all follow a common theme: normalize across spatial dimensions and discard the extracted statistics. In this paper, we propose a novel normalization method that deviates from this theme. Our approach, which we refer to as Positional Normalization (PONO), normalizes exclusively across channels, which allows us to capture structural information of the input image in the first and second moments. Instead of disregarding this information, we inject it into later layers to preserve or transfer structural information in generative networks. We show that PONO significantly improves the performance of deep networks across a wide range of model architectures and image generation tasks.
Live content is unavailable. Log in and register to view live content