Timezone: »

 
Spectral Tensor Layer for Model-Parallel Deep Neural Networks
Zhiyuan Wang · Xiao-Yang Liu

@
in Second Workshop on Quantum Tensor Networks in Machine Learning »

We propose a novel spectral tensor layer for model-parallel deep neural networks, which can be trainedin the spectrum domain. Spectral tensor neu-ral network is trained by training a number of small,weak subnetworks which can be trained in asyn-chronous parallel. The output of the spectral tensorlayer neural network is the ensemble of the subnet-works’ outputs. Compared with conventional neuralnetworks, spectral tensor neural network has intrin-sical data parallelism as well as model parallelism,which is very suitable for distributed training, and hascompetitive ensemble results.

Author Information

Zhiyuan Wang (Huazhong University of Science and Technology)
Xiao-Yang Liu (Columbia University)

More from the Same Authors