Timezone: »
Tensor-based multimodal fusion techniques have exhibited great predictive performance. However, one limitation is that existing approaches only consider bilinear or trilinear pooling, which fails to unleash the complete expressive power of multilinear fusion with restricted orders of interactions. More importantly, simply fusing features all at once ignores the complex local intercorrelations, leading to the deterioration of prediction. In this work, we first propose a polynomial tensor pooling (PTP) block for integrating multimodal features by considering high-order moments, followed by a tensorized fully connected layer. Treating PTP as a building block, we further establish a hierarchical polynomial fusion network (HPFN) to recursively transmit local correlations into global ones. By stacking multiple PTPs, the expressivity capacity of HPFN enjoys an exponential growth w.r.t. the number of layers, which is shown by the equivalence to a very deep convolutional arithmetic circuits. Various experiments demonstrate that it can achieve the state-of-the-art performance.
Author Information
Ming Hou (RIKEN AIP)
Jiajia Tang (Hangzhou Dianzi University / RIKEN AIP)
Jianhai Zhang (Hangzhou Dianzi University)
Wanzeng Kong (Hangzhou Dianzi University)
Qibin Zhao (RIKEN AIP)
More from the Same Authors
-
2021 : Bayesian Tensor Networks »
Kriton Konstantinidis · Yao Lei Xu · Qibin Zhao · Danilo Mandic -
2021 : Bayesian Latent Factor Model for Higher-order Data: an Extended Abstract »
Zerui Tao · Xuyang ZHAO · Toshihisa Tanaka · Qibin Zhao -
2021 : Is Rank Minimization of the Essence to Learn Tensor Network Structure? »
Chao Li · Qibin Zhao -
2021 : Fully-Connected Tensor Network Decomposition »
Yu-Bang Zheng · Ting-Zhu Huang · Xi-Le Zhao · Qibin Zhao · Tai-Xiang Jiang -
2022 Poster: SPD domain-specific batch normalization to crack interpretable unsupervised domain adaptation in EEG »
Reinmar Kobler · Jun-ichiro Hirayama · Qibin Zhao · Motoaki Kawanabe -
2021 : Discussion Pannel »
Xiao-Yang Liu · Qibin Zhao · Chao Li · Guillaume Rabusseau -
2021 : Bayesian Tensor Networks »
Kriton Konstantinidis · Yao Lei Xu · Qibin Zhao · Danilo Mandic -
2021 Workshop: Second Workshop on Quantum Tensor Networks in Machine Learning »
Xiao-Yang Liu · Qibin Zhao · Ivan Oseledets · Yufei Ding · Guillaume Rabusseau · Jean Kossaifi · Khadijeh Najafi · Anwar Walid · Andrzej Cichocki · Masashi Sugiyama -
2020 Workshop: First Workshop on Quantum Tensor Networks in Machine Learning »
Xiao-Yang Liu · Qibin Zhao · Jacob Biamonte · Cesar F Caiafa · Paul Pu Liang · Nadav Cohen · Stefan Leichenauer -
2011 Poster: A Multilinear Subspace Regression Method Using Orthogonal Tensors Decompositions »
Qibin Zhao · Cesar F Caiafa · Danilo Mandic · Liqing Zhang · Tonio Ball · Andreas Schulze-bonhage · Andrzej S CICHOCKI -
2011 Spotlight: A Multilinear Subspace Regression Method Using Orthogonal Tensors Decompositions »
Qibin Zhao · Cesar F Caiafa · Danilo Mandic · Liqing Zhang · Tonio Ball · Andreas Schulze-bonhage · Andrzej S CICHOCKI