Timezone: »
Poster
Matrix Inference and Estimation in MultiLayer Models
Parthe Pandit · Mojtaba Sahraee Ardakan · Sundeep Rangan · Philip Schniter · Alyson Fletcher
We consider the problem of estimating the input and hidden variables of a stochastic multilayer neural network from an observation of the output. The hidden variables in each layer are represented as matrices with statistical interactions along both rows as well as columns. This problem applies to matrix imputation, signal recovery via deep generative prior models, multitask and mixed regression, and learning certain classes of twolayer neural networks. We extend a recentlydeveloped algorithm  MultiLayer Vector Approximate Message Passing (MLVAMP), for this matrixvalued inference problem. It is shown that the performance of the proposed MultiLayer Matrix VAMP (MLMatVAMP) algorithm can be exactly predicted in a certain random largesystem limit, where the dimensions $N\times d$ of the unknown quantities grow as $N\rightarrow\infty$ with $d$ fixed. In the twolayer neuralnetwork learning problem, this scaling corresponds to the case where the number of input features, as well as training samples, grow to infinity but the number of hidden nodes stays fixed. The analysis enables a precise prediction of the parameter and test error of the learning.
Author Information
Parthe Pandit (University of California, Los Angeles)
Parthe is a Postdoc at the Halicioglu Data Science Institute at UCSD. He is interested in highdimensional optimization problems in machine learning and signal processing.
Mojtaba Sahraee Ardakan (UCLA)
Sundeep Rangan (NYU)
Philip Schniter (The Ohio State University)
Alyson Fletcher (UCLA)
More from the Same Authors

2022 Poster: Instability and Local Minima in GAN Training with Kernel Discriminators »
Evan Becker · Parthe Pandit · Sundeep Rangan · Alyson Fletcher 
2019 Poster: InputOutput Equivalence of Unitary and Contractive RNNs »
Melikasadat Emami · Mojtaba Sahraee Ardakan · Sundeep Rangan · Alyson Fletcher 
2018 Poster: Plugin Estimation in HighDimensional Linear Inverse Problems: A Rigorous Analysis »
Alyson Fletcher · Parthe Pandit · Sundeep Rangan · Subrata Sarkar · Philip Schniter 
2017 Poster: Rigorous Dynamics and Consistent Estimation in Arbitrarily Conditioned Linear Systems »
Alyson Fletcher · Mojtaba SahraeeArdakan · Sundeep Rangan · Philip Schniter 
2014 Poster: Scalable Inference for Neuronal Connectivity from Calcium Imaging »
Alyson Fletcher · Sundeep Rangan 
2014 Spotlight: Scalable Inference for Neuronal Connectivity from Calcium Imaging »
Alyson Fletcher · Sundeep Rangan 
2012 Poster: Approximate Message Passing with Consistent Parameter Estimation and Applications to Sparse Learning »
Ulugbek S Kamilov · Sundeep Rangan · Alyson Fletcher · MIchael Unser 
2011 Poster: Neural Reconstruction with Approximate Message Passing (NeuRAMP) »
Alyson Fletcher · Sundeep Rangan · Lav R Varshney · Aniruddha Bhargava