An extremely coarse feedback signal is sufficient for learning human-aligned visual representations
Abstract
Artificial neural networks trained on visual tasks develop internal representations that resemble those of the primate visual system. However, it is unclear how fine-grained the learning signal must be to produce these brain-aligned representations. We test this by training otherwise identical networks on ImageNet with label spaces ranging from 2 to 1,000 categories. The coarse labels are derived from broad axes of visual variation, while all other training conditions are held fixed. Networks trained with as few as eight broad categories match the neural alignment of 1,000-class models across macaque electrophysiology and human fMRI. This result does not arise from a loss of fine-scale structure. Across 100 held-out ImageNet categories, eight-class networks show nearly the same fine-category manifold signal-to-noise ratio as 1,000-class networks. These results show that fine-grained supervision is not necessary for learning either brain-aligned representations or separable fine-category geometry. Instead, a small number of meaningful visual distinctions can be sufficient to induce rich internal representations.