Globally Optimal Training of Generalized Polynomial Neural Networks with Nonlinear Spectral Methods
Antoine Gautier · Quynh N Nguyen · Matthias Hein

Tue Dec 6th 06:00 -- 09:30 PM @ Area 5+6+7+8 #98 #None

The optimization problem behind neural networks is highly non-convex. Training with stochastic gradient descent and variants requires careful parameter tuning and provides no guarantee to achieve the global optimum. In contrast we show under quite weak assumptions on the data that a particular class of feedforward neural networks can be trained globally optimal with a linear convergence rate. Up to our knowledge this is the first practically feasible method which achieves such a guarantee. While the method can in principle be applied to deep networks, we restrict ourselves for simplicity in this paper to one- and two hidden layer networks. Our experiments confirms that these models are already rich enough to achieve good performance on a series of real-world datasets.

Author Information

Antoine Gautier (Saarland University)
Quynh N Nguyen (Saarland University)
Matthias Hein (Saarland University)

More from the Same Authors