Timezone: »

 
Poster
NeRV: Neural Representations for Videos
Hao Chen · Bo He · Hanyu Wang · Yixuan Ren · Ser Nam Lim · Abhinav Shrivastava

Thu Dec 09 08:30 AM -- 10:00 AM (PST) @
in Poster Session 6 »
We propose a novel neural representation for videos (NeRV) which encodes videos in neural networks. Unlike conventional representations that treat videos as frame sequences, we represent videos as neural networks taking frame index as input. Given a frame index, NeRV outputs the corresponding RGB image. Video encoding in NeRV is simply fitting a neural network to video frames and decoding process is a simple feedforward operation. As an image-wise implicit representation, NeRV output the whole image and shows great efficiency compared to pixel-wise implicit representation, improving the encoding speed by $\textbf{25}\times$ to $\textbf{70}\times$, the decoding speed by $\textbf{38}\times$ to $\textbf{132}\times$, while achieving better video quality. With such a representation, we can treat videos as neural networks, simplifying several video-related tasks. For example, conventional video compression methods are restricted by a long and complex pipeline, specifically designed for the task. In contrast, with NeRV, we can use any neural network compression method as a proxy for video compression, and achieve comparable performance to traditional frame-based video compression approaches (H.264, HEVC \etc). Besides compression, we demonstrate the generalization of NeRV for video denoising. The source code and pre-trained model can be found at https://github.com/haochen-rye/NeRV.git.
Keywords: Deep Learning · Representation Learning

Author Information

Hao Chen (University of Maryland, College Park)
Bo He (University of Maryland, College Park)
Hanyu Wang (University of Maryland, College Park)
Yixuan Ren (University of Maryland)
Ser Nam Lim (Facebook AI)
Abhinav Shrivastava (University of Maryland, College Park)

More from the Same Authors