Timezone: »

 
Poster
Paraphrase Generation with Latent Bag of Words
Yao Fu · Yansong Feng · John Cunningham

Thu Dec 12 05:00 PM -- 07:00 PM (PST) @ East Exhibition Hall B + C #139
in Applications -- Natural Language Processing »

Paraphrase generation is a longstanding important problem in natural language processing. Recent progress in deep generative models has shown promising results on discrete latent variables for text generation. Inspired by variational autoencoders with discrete latent structures, in this work, we propose a latent bag of words (BOW) model for paraphrase generation. We ground the semantics of a discrete latent variable by the target BOW. We use this latent variable to build a fully differentiable content planning and surface realization pipeline. Specifically, we use source words to predict their neighbors and model the target BOW with a mixture of softmax. We use gumbel top-k reparameterization to perform differentiable subset sampling from the predicted BOW distribution. We retrieve the sampled word embeddings and use them to augment the decoder and guide its generation search space. Our latent BOW model not only enhances the decoder, but also exhibits clear interpretability. We show the model interpretability with regard to (1). unsupervised learning of word neighbors (2). the step-by-step generation procedure. Extensive experiments demonstrate the model's transparent and effective generation process.

Keywords: Natural Language Processing · Latent Variable Models · Applications · Deep Learning -> Generative Models; Probabilistic Methods
[ Paper]  [ Slides] 

Author Information

Yao Fu (Columbia University)
Yansong Feng (Peking University)
John Cunningham (University of Columbia)

More from the Same Authors