Timezone: »

Compete to Compute
Rupesh K Srivastava · Jonathan Masci · Sohrob Kazerounian · Faustino Gomez · Jürgen Schmidhuber

Fri Dec 06 07:00 PM -- 11:59 PM (PST) @ Harrah's Special Events Center, 2nd Floor #None

Local competition among neighboring neurons is common in biological neural networks (NNs). We apply the concept to gradient-based, backprop-trained artificial multilayer NNs. NNs with competing linear units tend to outperform those with non-competing nonlinear units, and avoid catastrophic forgetting when training sets change over time.

Author Information

Rupesh K Srivastava (NNAISENSE)
Jonathan Masci (Università della Svizzera italiana)
Sohrob Kazerounian (IDSIA)
Faustino Gomez (IDSIA)
Jürgen Schmidhuber (Swiss AI Lab, IDSIA (USI & SUPSI) - NNAISENSE)

Since age 15, his main goal has been to build an Artificial Intelligence smarter than himself, then retire. The Deep Learning Artificial Neural Networks developed since 1991 by his research groups have revolutionised handwriting recognition, speech recognition, machine translation, image captioning, and are now available to billions of users through Google, Microsoft, IBM, Baidu, and many other companies (DeepMind also was heavily influenced by his lab). His team's Deep Learners were the first to win object detection and image segmentation contests, and achieved the world's first superhuman visual classification results, winning nine international competitions in machine learning & pattern recognition. His formal theory of fun & creativity & curiosity explains art, science, music, and humor. He has published 333 papers, earned 7 best paper/best video awards, the 2013 Helmholtz Award of the International Neural Networks Society, and the 2016 IEEE Neural Networks Pioneer Award. He is also president of NNAISENSE, which aims at building the first practical general purpose AI.

More from the Same Authors