Poster
On the Computational Efficiency of Training Neural Networks
Roi Livni · Shai Shalev-Shwartz · Ohad Shamir

Mon Dec 8th 07:00 -- 11:59 PM @ Level 2, room 210D #None
in Poster Session and Reception »

It is well-known that neural networks are computationally hard to train. On the other hand, in practice, modern day neural networks are trained efficiently using SGD and a variety of tricks that include different activation functions (e.g. ReLU), over-specification (i.e., train networks which are larger than needed), and regularization. In this paper we revisit the computational complexity of training neural networks from a modern perspective. We provide both positive and negative results, some of them yield new provably efficient and practical algorithms for training neural networks.

Author Information

Roi Livni (Hebrew University)
Shai Shalev-Shwartz (Mobileye & HUJI)
Ohad Shamir (Weizmann Institute of Science)

More from the Same Authors