Timezone: »

 
An Empirical Analysis of the Advantages of Finite vs.~Infinite Width Bayesian Neural Networks
Jiayu Yao · Yaniv Yacoby · Beau Coker · Weiwei Pan · Finale Doshi-Velez

@
in Gaussian Processes, Spatiotemporal Modeling, and Decision-making Systems »

Comparing Bayesian neural networks (BNNs) with different widths is challenging because, as the width increases, multiple model properties change simultaneously, and, inference in the finite width case is intractable. In this work, we empirically compare finite and infinite width BNNs, and provide quantitative and qualitative explanations for their performance difference. We find that under model mis-specification, increasing width can hurt BNN performance. In these cases, we provide evidence that finite BNNs generalize better partially due to the properties of their frequency spectrum that allows them to adapt under model mismatch.

[ Poster] 

Author Information

Jiayu Yao (Harvard University)
Yaniv Yacoby (Harvard University)
Beau Coker (Harvard)
Weiwei Pan (Harvard University)
Finale Doshi-Velez (Harvard)

More from the Same Authors