Timezone: »
Spectral Tensor Layer for Model-Parallel Deep Neural Networks
Zhiyuan Wang · Xiao-Yang Liu
We propose a novel spectral tensor layer for model-parallel deep neural networks, which can be trainedin the spectrum domain. Spectral tensor neu-ral network is trained by training a number of small,weak subnetworks which can be trained in asyn-chronous parallel. The output of the spectral tensorlayer neural network is the ensemble of the subnet-works’ outputs. Compared with conventional neuralnetworks, spectral tensor neural network has intrin-sical data parallelism as well as model parallelism,which is very suitable for distributed training, and hascompetitive ensemble results.
Author Information
Zhiyuan Wang (Huazhong University of Science and Technology)
Xiao-Yang Liu (Columbia University)
More from the Same Authors
-
2021 : GPU-Podracer: Scalable and Elastic Library for Cloud-Native Deep Reinforcement Learning »
Xiao-Yang Liu · Zhuoran Yang · Zhaoran Wang · Anwar Walid · Jian Guo · Michael Jordan -
2021 : Graph-Tensor Singular Value Decomposition for Data Recovery »
Lei Deng · Haifeng Zheng · Xiao-Yang Liu -
2021 : High Performance Hierarchical Tucker Tensor Learning Using GPU Tensor Cores »
hao huang · Xiao-Yang Liu · Weiqin Tong · Tao Zhang · Anwar Walid -
2021 : Codee: A Tensor Embedding Scheme for Binary Code Search »
Jia Yang · Cai Fu · Xiao-Yang Liu -
2021 : Deep variational reinforcement learning by optimizing Hamiltonian equation »
Zeliang Zhang · Xiao-Yang Liu -
2022 Poster: Homomorphic Matrix Completion »
Xiao-Yang Liu · Zechu (Steven) Li · Xiaodong Wang -
2022 Poster: FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learning »
Xiao-Yang Liu · Ziyi Xia · Jingyang Rui · Jiechao Gao · Hongyang Yang · Ming Zhu · Christina Wang · Zhaoran Wang · Jian Guo -
2021 : Discussion Pannel »
Xiao-Yang Liu · Qibin Zhao · Chao Li · Guillaume Rabusseau -
2021 : High Performance Computation for Tensor Networks Learning »
Anwar Walid · Xiao-Yang Liu -
2021 Workshop: Second Workshop on Quantum Tensor Networks in Machine Learning »
Xiao-Yang Liu · Qibin Zhao · Ivan Oseledets · Yufei Ding · Guillaume Rabusseau · Jean Kossaifi · Khadijeh Najafi · Anwar Walid · Andrzej Cichocki · Masashi Sugiyama -
2021 : Opening Remarks »
Xiao-Yang Liu -
2020 : Closing Remarks »
Xiao-Yang Liu -
2020 : Panel Discussion 2: Software and High Performance Implementation »
Glen Evenbly · Martin Ganahl · Paul Springer · Xiao-Yang Liu -
2020 : Panel Discussion 1: Theoretical, Algorithmic and Physical »
Jacob Biamonte · Ivan Oseledets · Jens Eisert · Nadav Cohen · Guillaume Rabusseau · Xiao-Yang Liu -
2020 Workshop: First Workshop on Quantum Tensor Networks in Machine Learning »
Xiao-Yang Liu · Qibin Zhao · Jacob Biamonte · Cesar F Caiafa · Paul Pu Liang · Nadav Cohen · Stefan Leichenauer -
2020 : Opening Remarks »
Xiao-Yang Liu -
2019 : Coffee + Posters »
Changhao Chen · Nils Gählert · Edouard Leurent · Johannes Lehner · Apratim Bhattacharyya · Harkirat Singh Behl · Teck Yian Lim · Shiho Kim · Jelena Novosel · Błażej Osiński · Arindam Das · Ruobing Shen · Jeffrey Hawke · Joachim Sicking · Babak Shahian Jahromi · Theja Tulabandhula · Claudio Michaelis · Evgenia Rusak · WENHANG BAO · Hazem Rashed · JP Chen · Amin Ansari · Jaekwang Cha · Mohamed Zahran · Daniele Reda · Jinhyuk Kim · Kim Dohyun · Ho Suk · Junekyo Jhung · Alexander Kister · Matthias Fahrland · Adam Jakubowski · Piotr Miłoś · Jean Mercat · Bruno Arsenali · Silviu Homoceanu · Xiao-Yang Liu · Philip Torr · Ahmad El Sallab · Ibrahim Sobh · Anurag Arnab · Krzysztof Galias -
2018 : Posters and Open Discussions (see below for poster titles) »
Ramya Malur Srinivasan · Miguel Perez · Yuanyuan Liu · Ben Wood · Dan Philps · Kyle Brown · Daniel Martin · Mykola Pechenizkiy · Luca Costabello · Rongguang Wang · Suproteem Sarkar · Sangwoong Yoon · Zhuoran Xiong · Enguerrand Horel · Zhu (Drew) Zhang · Ulf Johansson · Jonathan Kochems · Gregory Sidier · Prashant Reddy · Lana Cuthbertson · Yvonne Wambui · Christelle Marfaing · Galen Harrison · Irene Unceta Mendieta · Thomas Kehler · Mark Weber · Li Ling · Ceena Modarres · Abhinav Dhall · Arash Nourian · David Byrd · Ajay Chander · Xiao-Yang Liu · Hongyang Yang · Shuang (Sophie) Zhai · Freddy Lecue · Sirui Yao · Rory McGrath · Artur Garcez · Vangelis Bacoyannis · Alexandre Garcia · Lukas Gonon · Mark Ibrahim · Melissa Louie · Omid Ardakanian · Cecilia Sönströd · Kojin Oshiba · Chaofan Chen · Suchen Jin · aldo pareja · Toyo Suzumura