Timezone: »

 
Poster
Fast Structured Decoding for Sequence Models
Zhiqing Sun · Zhuohan Li · Haoqing Wang · Di He · Zi Lin · Zhihong Deng

Wed Dec 11 05:00 PM -- 07:00 PM (PST) @ East Exhibition Hall B + C #109
in Applications -- Natural Language Processing »

Autoregressive sequence models achieve state-of-the-art performance in domains like machine translation. However, due to the autoregressive factorization nature, these models suffer from heavy latency during inference. Recently, non-autoregressive sequence models were proposed to reduce the inference time. However, these models assume that the decoding process of each token is conditionally independent of others. Such a generation process sometimes makes the output sentence inconsistent, and thus the learned non-autoregressive models could only achieve inferior accuracy compared to their autoregressive counterparts. To improve then decoding consistency and reduce the inference cost at the same time, we propose to incorporate a structured inference module into the non-autoregressive models. Specifically, we design an efficient approximation for Conditional Random Fields (CRF) for non-autoregressive sequence models, and further propose a dynamic transition technique to model positional contexts in the CRF. Experiments in machine translation show that while increasing little latency (8~14ms, our model could achieve significantly better translation performance than previous non-autoregressive models on different translation datasets. In particular, for the WMT14 En-De dataset, our model obtains a BLEU score of 26.80, which largely outperforms the previous non-autoregressive baselines and is only 0.61 lower in BLEU than purely autoregressive models.

Keywords: Attention Models · Natural Language Processing · Applications · Deep Learning
[ Paper]  [ Poster] 

Author Information

Zhiqing Sun (Carnegie Mellon University)
Zhuohan Li (UC Berkeley)
Haoqing Wang (Peking University)
Di He (Peking University)
Zi Lin (Peking University)
Zhihong Deng (Peking University)

More from the Same Authors