Skip to yearly menu bar Skip to main content


The Diversified Ensemble Neural Network

Shaofeng Zhang · Meng Liu · Junchi Yan

Poster Session 3 #1114

Keywords: [ Applications ] [ Matrix and Tensor Factorization ] [ Algorithms -> Unsupervised Learning; Probabilistic Methods -> Graphical Models; Probabilistic Methods ] [ Latent Variable Models ]


Ensemble is a general way of improving the accuracy and stability of learning models, especially for the generalization ability on small datasets. Compared with tree-based methods, relatively less works have been devoted to an in-depth study on effective ensemble design for neural networks. In this paper, we propose a principled ensemble technique by constructing the so-called diversified ensemble layer to combine multiple networks as individual modules. We theoretically show that each individual model in our ensemble layer corresponds to weights in the ensemble layer optimized in different directions. Meanwhile, the devised ensemble layer can be readily integrated into popular neural architectures, including CNNs, RNNs, and GCNs. Extensive experiments are conducted on public tabular datasets, images, and texts. By adopting weight sharing approach, the results show our method can notably improve the accuracy and stability of the original neural networks with ignorable extra time and space overhead.

Chat is not available.