Timezone: »
The usage of deep generative models for image compression has led to impressive performance gains over classical codecs while neural video compression is still in its infancy. Here, we propose an end-to-end, deep generative modeling approach to compress temporal sequences with a focus on video. Our approach builds upon variational autoencoder (VAE) models for sequential data and combines them with recent work on neural image compression. The approach jointly learns to transform the original sequence into a lower-dimensional representation as well as to discretize and entropy code this representation according to predictions of the sequential VAE. Rate-distortion evaluations on small videos from public data sets with varying complexity and diversity show that our model yields competitive results when trained on generic video content. Extreme compression performance is achieved when training the model on specialized content.
Author Information
Salvator Lombardo (Disney Research)
JUN HAN (Dartmouth College)
I am a Ph.D. student in Computer Science at Dartmouth College.
Christopher Schroers (Disney Research|Studios)
Stephan Mandt (University of California, Irvine)

Stephan Mandt is an Associate Professor of Computer Science and Statistics at the University of California, Irvine. From 2016 until 2018, he was a Senior Researcher and Head of the statistical machine learning group at Disney Research in Pittsburgh and Los Angeles. He held previous postdoctoral positions at Columbia University and Princeton University. Stephan holds a Ph.D. in Theoretical Physics from the University of Cologne, where he received the German National Merit Scholarship. He is furthermore a recipient of the NSF CAREER Award, the UCI ICS Mid-Career Excellence in Research Award, the German Research Foundation's Mercator Fellowship, a Kavli Fellow of the U.S. National Academy of Sciences, a member of the ELLIS Society, and a former visiting researcher at Google Brain. Stephan regularly serves as an Area Chair, Action Editor, or Editorial Board member for NeurIPS, ICML, AAAI, ICLR, TMLR, and JMLR. His research is currently supported by NSF, DARPA, DOE, Disney, Intel, and Qualcomm.
More from the Same Authors
-
2021 : Analyzing High-Resolution Clouds and Convection using Multi-Channel VAEs »
Harshini Mangipudi · Griffin Mooers · Mike Pritchard · Tom Beucler · Stephan Mandt -
2021 : Structured Stochastic Gradient MCMC: a hybrid VI and MCMC approach »
Antonios Alexos · Alex Boyd · Stephan Mandt -
2022 : Probabilistic Querying of Continuous-Time Sequential Events »
Alex Boyd · Yuxin Chang · Stephan Mandt · Padhraic Smyth -
2022 : An Unsupervised Learning Perspective on the Dynamic Contribution to Extreme Precipitation Changes »
Griffin Mooers · Tom Beucler · Mike Pritchard · Stephan Mandt -
2022 : Panel »
Virginia Smith · Michele Covell · Daniel Severo · Christopher Schroers -
2022 : Q & A »
Karen Ullrich · Yibo Yang · Stephan Mandt -
2022 Tutorial: Data Compression with Machine Learning »
Karen Ullrich · Yibo Yang · Stephan Mandt -
2022 : Tutorial part 1 »
Yibo Yang · Karen Ullrich · Stephan Mandt -
2022 Poster: Predictive Querying for Autoregressive Neural Sequence Models »
Alex Boyd · Samuel Showalter · Stephan Mandt · Padhraic Smyth -
2021 Poster: Detecting and Adapting to Irregular Distribution Shifts in Bayesian Online Learning »
Aodong Li · Alex Boyd · Padhraic Smyth · Stephan Mandt -
2020 : Q/A and Discussion for ML Theory Session »
Karthik Kashinath · Mayur Mudigonda · Stephan Mandt · Rose Yu -
2020 : Stephan Mandt »
Stephan Mandt -
2020 Poster: User-Dependent Neural Sequence Models for Continuous-Time Event Data »
Alex Boyd · Robert Bamler · Stephan Mandt · Padhraic Smyth -
2020 Poster: Improving Inference for Neural Image Compression »
Yibo Yang · Robert Bamler · Stephan Mandt -
2017 : Introduction »
Cheng Zhang · Francisco Ruiz · Dustin Tran · James McInerney · Stephan Mandt -
2017 Workshop: Advances in Approximate Bayesian Inference »
Francisco Ruiz · Stephan Mandt · Cheng Zhang · James McInerney · James McInerney · Dustin Tran · Dustin Tran · David Blei · Max Welling · Tamara Broderick · Michalis Titsias -
2017 Poster: Perturbative Black Box Variational Inference »
Robert Bamler · Cheng Zhang · Manfred Opper · Stephan Mandt -
2016 Workshop: Advances in Approximate Bayesian Inference »
Tamara Broderick · Stephan Mandt · James McInerney · Dustin Tran · David Blei · Kevin Murphy · Andrew Gelman · Michael I Jordan -
2016 Poster: Exponential Family Embeddings »
Maja Rudolph · Francisco Ruiz · Stephan Mandt · David Blei -
2016 Poster: Bootstrap Model Aggregation for Distributed Statistical Learning »
JUN HAN · Qiang Liu -
2015 : Finding Sparse Features in Strongly Confounded Medial Data »
Stephan Mandt · Florian Wenzel -
2015 Workshop: Advances in Approximate Bayesian Inference »
Dustin Tran · Tamara Broderick · Stephan Mandt · James McInerney · Shakir Mohamed · Alp Kucukelbir · Matthew D. Hoffman · Neil Lawrence · David Blei -
2014 Poster: Smoothed Gradients for Stochastic Variational Inference »
Stephan Mandt · David Blei