Learning Flexible Classifiers with Shot-CONditional Episodic (SCONE) Training

Early few-shot classification work advocates for episodic training, i.e. training over learning episodes each posing a few-shot classification task. However, the role of this training regime remains poorly understood. Standard classification methods (pre-training'') followed by episodic fine-tuning have recently achieved strong results. We aim to understand the role of this episodic fine-tuning phase through an exploration of the effect of theshot'' (number of examples per class) that is used during fine-tuning. We discover that using a fixed shot can specialize the pre-trained model to solving episodes of that shot at the expense of performance on other shots, in agreement with a trade-off recently observed in the context of end-to-end episodic training. To amend this, we propose a shot-conditional form of episodic fine-tuning, inspired from recent work that trains a single model on a distribution of losses. We show that this flexible approach consitutes an effective general solution that does not suffer disproportionately on any shot. We then subject it to the large-scale Meta-Dataset benchmark of varying shots and imbalanced episodes and observe performance gains in that challenging environment.