`

Timezone: »

 
Poster
On the Computational Efficiency of Training Neural Networks
Roi Livni · Shai Shalev-Shwartz · Ohad Shamir

Mon Dec 08 04:00 PM -- 08:59 PM (PST) @ Level 2, room 210D #None
in Poster Session and Reception »

It is well-known that neural networks are computationally hard to train. On the other hand, in practice, modern day neural networks are trained efficiently using SGD and a variety of tricks that include different activation functions (e.g. ReLU), over-specification (i.e., train networks which are larger than needed), and regularization. In this paper we revisit the computational complexity of training neural networks from a modern perspective. We provide both positive and negative results, some of them yield new provably efficient and practical algorithms for training neural networks.

[ PDF] 

Author Information

Roi Livni (Hebrew University)
Shai Shalev-Shwartz (Mobileye & HUJI)
Ohad Shamir (Weizmann Institute of Science)

More from the Same Authors