Timezone: »

 
Incremental Fourier Neural Operator
Jiawei Zhao · Robert Joseph George · Yifei Zhang · Zongyi Li · Anima Anandkumar

@
in AI for Science: Progress and Promises »
Event URL: https://openreview.net/forum?id=duCmFUWpxVj »

Recently, neural networks have proven their impressive ability to solve partial differential equations (PDEs). Among them, Fourier neural operator (FNO) has shown success in learning solution operators for highly non-linear problems such as turbulence flow. FNO is discretization-invariant, where it can be trained on low-resolution data and generalizes to problems with high-resolution. This property is related to the low-pass filters in FNO, where only a limited number of frequency modes are selected to propagate information. However, it is still a challenge to select an appropriate number of frequency modes and training resolution for different PDEs. Too few frequency modes and low-resolution data hurt generalization, while too many frequency modes and high-resolution data are computationally expensive and lead to over-fitting. To this end, we propose Incremental Fourier Neural Operator (IFNO), which augments both the frequency modes and data resolution incrementally during training. We show that IFNO achieves better generalization (around 15% reduction on testing L2 loss) while reducing the computational cost by 35%, compared to the standard FNO. In addition, we observe that IFNO follows the behavior of implicit regularization in FNO, which explains its excellent generalization ability.

Keywords: generalization · Operator Learning · implicit regularization

Author Information

Jiawei Zhao (Caltech)
Robert Joseph George (University of Alberta)
Yifei Zhang (University of Wisconsin - Madison)
Zongyi Li (Washington University in St. Louis)
Anima Anandkumar (NVIDIA / Caltech)

More from the Same Authors