Exploring the Inductive Bias from the Positional Encoding of Neural Network Reparametrization Methods for Geophysical Inversion

In geophysical inverse problems, we seek a geologically reasonable geophysical model whose corresponding predicted geophysical data matches the observed geophysical data. The forward model (PDE-based simulator) maps the subsurface properties (e.g., density, resistivity, wave velocity) to the predicted data at the surface. The inversion then minimizes the difference between observed and predicted data. The geophysical inverse problem is ill-posed; therefore, inverse modelling focuses on designing a regularization to recover geologically plausible models. Neural network reparameterization (NNR) methods change the optimization space from the mesh to the weights of a neural network. Unlike supervised learning, NNR leverages the implicit regularization of ML architectures and optimizers without requiring training data. The weights of the ML model are learned during each inversion process. We find that the positional encoding has an inductive bias that affects the convergence rate and recovered features of the inverted geophysical model. We test our hypothesis in multiple synthetic cases, including the three-dimensional (3D) muon tomography surveys and the two-dimensional (2D) seismic tomography surveys.