Timezone: »

 
Poster
Towards Text Generation with Adversarially Learned Neural Outlines
Sandeep Subramanian · Sai Rajeswar Mudumba · Alessandro Sordoni · Adam Trischler · Aaron Courville · Chris Pal

Tue Dec 04 07:45 AM -- 09:45 AM (PST) @ Room 210 #14
in Tue Poster Session A »

Recent progress in deep generative models has been fueled by two paradigms -- autoregressive and adversarial models. We propose a combination of both approaches with the goal of learning generative models of text. Our method first produces a high-level sentence outline and then generates words sequentially, conditioning on both the outline and the previous outputs. We generate outlines with an adversarial model trained to approximate the distribution of sentences in a latent space induced by general-purpose sentence encoders. This provides strong, informative conditioning for the autoregressive stage. Our quantitative evaluations suggests that conditioning information from generated outlines is able to guide the autoregressive model to produce realistic samples, comparable to maximum-likelihood trained language models, even at high temperatures with multinomial sampling. Qualitative results also demonstrate that this generative procedure yields natural-looking sentences and interpolations.

Keywords: Adversarial Networks · Generative Models · Representation Learning · Natural Language Processing
[ Paper] 

Author Information

Sandeep Subramanian (University of Montreal)
Sai Rajeswar Mudumba (University of Montreal)
Alessandro Sordoni (Microsoft Research Montreal)
Adam Trischler (Microsoft)
Aaron Courville (U. Montreal)
Chris Pal (MILA, Polytechnique Montréal, Element AI)

More from the Same Authors