Poster
MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis
Kundan Kumar · Rithesh Kumar · Thibault de Boissiere · Lucas Gestin · Wei Zhen Teoh · Jose Sotelo · Alexandre de Brébisson · Yoshua Bengio · Aaron Courville

Wed Dec 11th 05:00 -- 07:00 PM @ East Exhibition Hall B + C #77
in Applications -- Audio and Speech Processing »

Previous works (Donahue et al., 2018a; Engel et al., 2019a) have found that generating coherent raw audio waveforms with GANs is challenging. In this paper, we show that it is possible to train GANs reliably to generate high quality coherent waveforms by introducing a set of architectural changes and simple training techniques. Subjective evaluation metric (Mean Opinion Score, or MOS) shows the effectiveness of the proposed approach for high quality mel-spectrogram inversion. To establish the generality of the proposed techniques, we show qualitative results of our model in speech synthesis, music domain translation and unconditional music synthesis. We evaluate the various components of the model through ablation studies and suggest a set of guidelines to design general purpose discriminators and generators for conditional sequence synthesis tasks. Our model is non-autoregressive, fully convolutional, with significantly fewer parameters than competing models and generalizes to unseen speakers for mel-spectrogram inversion. Our pytorch implementation runs at more than 100x faster than realtime on GTX 1080Ti GPU and more than 2x faster than real-time on CPU, without any hardware specific optimization tricks.

Keywords: Applications -> Audio and Speech Processing · Applications -> Music Modeling and Analysis; Deep Learning -> Adversarial Networks; Deep Learning -> Generative Models

Author Information

Kundan Kumar (Lyrebird-AI, Mila)

Phd Student, MILA, Universite de Montreal Co-founder, Lyrebird-AI

Rithesh Kumar (Mila / Lyrebird)
Thibault de Boissiere (Lyrebird)
Lucas Gestin (Lyrebird)
Wei Zhen Teoh (Lyrebird)
Jose Sotelo (MILA, Lyrebird)
Alexandre de Brébisson (LYREBIRD, MILA)
Yoshua Bengio (Mila)

Yoshua Bengio is Full Professor in the computer science and operations research department at U. Montreal, scientific director and founder of Mila and of IVADO, Turing Award 2018 recipient, Canada Research Chair in Statistical Learning Algorithms, as well as a Canada AI CIFAR Chair. He pioneered deep learning and has been getting the most citations per day in 2018 among all computer scientists, worldwide. He is an officer of the Order of Canada, member of the Royal Society of Canada, was awarded the Killam Prize, the Marie-Victorin Prize and the Radio-Canada Scientist of the year in 2017, and he is a member of the NeurIPS advisory board and co-founder of the ICLR conference, as well as program director of the CIFAR program on Learning in Machines and Brains. His goal is to contribute to uncover the principles giving rise to intelligence through learning, as well as favour the development of AI for the benefit of all.

Aaron Courville (U. Montreal)

More from the Same Authors