Timezone: »
We show how transformers can be used to vastly simplify neural video compression. Previous methods have been relying on an increasing number of architectural biases and priors, including motion prediction and warping operations, resulting in complex models. Instead, we independently map input frames to representations and use a transformer to model their dependencies, letting it predict the distribution of future representations given the past. The resulting video compression transformer outperforms previous methods on standard video compression data sets. Experiments on synthetic data show that our model learns to handle complex motion patterns such as panning, blurring and fading purely from data. Our approach is easy to implement, and we release code to facilitate future research.
Author Information
Fabian Mentzer (Google)
George D Toderici (Google)
David Minnen (Google)
Sergi Caelles (Google)
Sung Jin Hwang (Google)
Mario Lucic (Google Brain)
Eirikur Agustsson (Google)
More from the Same Authors
-
2022 Poster: Object Scene Representation Transformer »
Mehdi S. M. Sajjadi · Daniel Duckworth · Aravindh Mahendran · Sjoerd van Steenkiste · Filip Pavetic · Mario Lucic · Leonidas Guibas · Klaus Greff · Thomas Kipf -
2021 Poster: A Near-Optimal Algorithm for Debiasing Trained Machine Learning Models »
Ibrahim Alabdulmohsin · Mario Lucic -
2021 Poster: MLP-Mixer: An all-MLP Architecture for Vision »
Ilya Tolstikhin · Neil Houlsby · Alexander Kolesnikov · Lucas Beyer · Xiaohua Zhai · Thomas Unterthiner · Jessica Yung · Andreas Steiner · Daniel Keysers · Jakob Uszkoreit · Mario Lucic · Alexey Dosovitskiy -
2021 Poster: Revisiting the Calibration of Modern Neural Networks »
Matthias Minderer · Josip Djolonga · Rob Romijnders · Frances Hubis · Xiaohua Zhai · Neil Houlsby · Dustin Tran · Mario Lucic -
2020 Poster: High-Fidelity Generative Image Compression »
Fabian Mentzer · George D Toderici · Michael Tschannen · Eirikur Agustsson -
2020 Oral: High-Fidelity Generative Image Compression »
Fabian Mentzer · George D Toderici · Michael Tschannen · Eirikur Agustsson -
2020 Session: Orals & Spotlights Track 08: Deep Learning »
Graham Taylor · Mario Lucic -
2018 Poster: Deep Generative Models for Distribution-Preserving Lossy Compression »
Michael Tschannen · Eirikur Agustsson · Mario Lucic -
2018 Poster: Joint Autoregressive and Hierarchical Priors for Learned Image Compression »
David Minnen · Johannes Ballé · Johannes Ballé · George D Toderici -
2018 Poster: Assessing Generative Models via Precision and Recall »
Mehdi S. M. Sajjadi · Olivier Bachem · Mario Lucic · Olivier Bousquet · Sylvain Gelly -
2018 Poster: Are GANs Created Equal? A Large-Scale Study »
Mario Lucic · Karol Kurach · Marcin Michalski · Sylvain Gelly · Olivier Bousquet