Timezone: »

 
Poster
The Diversified Ensemble Neural Network
Shaofeng Zhang · Meng Liu · Junchi Yan

Wed Dec 09 09:00 AM -- 11:00 AM (PST) @ Poster Session 3 #1114
in Poster Session 4 »

Ensemble is a general way of improving the accuracy and stability of learning models, especially for the generalization ability on small datasets. Compared with tree-based methods, relatively less works have been devoted to an in-depth study on effective ensemble design for neural networks. In this paper, we propose a principled ensemble technique by constructing the so-called diversified ensemble layer to combine multiple networks as individual modules. We theoretically show that each individual model in our ensemble layer corresponds to weights in the ensemble layer optimized in different directions. Meanwhile, the devised ensemble layer can be readily integrated into popular neural architectures, including CNNs, RNNs, and GCNs. Extensive experiments are conducted on public tabular datasets, images, and texts. By adopting weight sharing approach, the results show our method can notably improve the accuracy and stability of the original neural networks with ignorable extra time and space overhead.

Keywords: Latent Variable Models · Matrix and Tensor Factorization · Applications · Algorithms -> Unsupervised Learning; Probabilistic Methods -> Graphical Models; Probabilistic Methods

Author Information

Shaofeng Zhang (University of Electronic Science and Technology of China)
Meng Liu (university of electronic science and technology of china)
Junchi Yan (Shanghai Jiao Tong University)

More from the Same Authors