Timezone: »
Dynamic neural networks toolkits such as PyTorch, DyNet, and Chainer offer more flexibility for implementing models that cope with data of varying dimensions and structure, relative to toolkits that operate on statically declared computations (e.g., TensorFlow, CNTK, and Theano). However, existing toolkits - both static and dynamic - require that the developer organize the computations into the batches necessary for exploiting high-performance data-parallel algorithms and hardware. This batching task is generally difficult, but it becomes a major hurdle as architectures become complex. In this paper, we present an algorithm, and its implementation in the DyNet toolkit, for automatically batching operations. Developers simply write minibatch computations as aggregations of single instance computations, and the batching algorithm seamlessly executes them, on the fly, in computationally efficient batches. On a variety of tasks, we obtain throughput similar to manual batches, as well as comparable speedups over single-instance learning on architectures that are impractical to batch manually.
Author Information
Graham Neubig (Carnegie Mellon University)
Yoav Goldberg (Bar-Ilan University)
Chris Dyer (DeepMind)
More from the Same Authors
-
2020 Poster: Learning Sparse Prototypes for Text Generation »
Junxian He · Taylor Berg-Kirkpatrick · Graham Neubig -
2019 Poster: Are Sixteen Heads Really Better than One? »
Paul Michel · Omer Levy · Graham Neubig -
2019 Poster: A Little Is Enough: Circumventing Defenses For Distributed Learning »
Moran Baruch · Gilad Baruch · Yoav Goldberg -
2017 Poster: Controllable Invariance through Adversarial Feature Learning »
Qizhe Xie · Zihang Dai · Yulun Du · Eduard Hovy · Graham Neubig -
2014 Workshop: Modern Machine Learning and Natural Language Processing »
Ankur P Parikh · Avneesh Saluja · Chris Dyer · Eric Xing -
2014 Poster: Neural Word Embedding as Implicit Matrix Factorization »
Omer Levy · Yoav Goldberg -
2014 Poster: Conditional Random Field Autoencoders for Unsupervised Structured Prediction »
Waleed Ammar · Chris Dyer · Noah A Smith -
2014 Oral: Conditional Random Field Autoencoders for Unsupervised Structured Prediction »
Waleed Ammar · Chris Dyer · Noah A Smith