Timezone: »

 
Poster
Understanding Benign Overfitting in Gradient-Based Meta Learning
Lisha Chen · Songtao Lu · Tianyi Chen

Wed Nov 30 09:00 AM -- 11:00 AM (PST) @ Hall J #206
in Poster Session 3 »

Meta learning has demonstrated tremendous success in few-shot learning with limited supervised data. In those settings, the meta model is usually overparameterized. While the conventional statistical learning theory suggests that overparameterized models tend to overfit, empirical evidence reveals that overparameterized meta learning methods still work well -- a phenomenon often called ``benign overfitting.'' To understand this phenomenon, we focus on the meta learning settings with a challenging bilevel structure that we term the gradient-based meta learning, and analyze its generalization performance under an overparameterized meta linear regression model. While our analysis uses the relatively tractable linear models, our theory contributes to understanding the delicate interplay among data heterogeneity, model adaptation and benign overfitting in gradient-based meta learning tasks. We corroborate our theoretical claims through numerical simulations.

Keywords: meta learning · Excess risk · benign overfitting
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Lisha Chen (Rensselaer Polytechnic Institute)
Songtao Lu (IBM Thomas J. Watson Research Center)
Tianyi Chen (Rensselaer Polytechnic Institute)

More from the Same Authors