Universal Functional Regression with Neural Operator Flows

Regression on function spaces is typically limited to models with Gaussian process priors. We introduce the notion of universal functional regression, in which we aim to learn a prior distribution over non-Gaussian function spaces that remains mathematically tractable for functional regression. To do this, we develop Neural Operator Flows (OpFlow), an infinite-dimensional extension of normalizing flows. OpFlow is an invertible operator that maps the data space into a Gaussian process, allowing for exact likelihood estimation of point evaluations of functions. OpFlow enables robust and accurate uncertainty quantification via drawing posterior samples. We empirically study the performance of OpFlow on regression tasks with data generated from Gaussian processes with known closed-form posterior distribution as well as highly non-Gaussian real-world earthquake time-series with an unknown closed-form posterior distribution.