Skip to yearly menu bar Skip to main content

Workshop: UniReps: Unifying Representations in Neural Models

On consequences of finetuning on data with highly discriminative features

Wojciech Masarczyk · Tomasz Trzcinski · Mateusz Ostaszewski

[ ] [ Project Page ]
presentation: UniReps: Unifying Representations in Neural Models
Fri 15 Dec 6:15 a.m. PST — 3:15 p.m. PST


In the era of transfer learning, training neural networks from scratch is becoming obsolete. Transfer learning leverages prior knowledge for new tasks, conserving computational resources. While its advantages are well-documented, we uncover a notable drawback: networks tend to prioritize basic data patterns, forsaking valuable pre-learned features. We term this behavior "feature erosion" and analyze its impact on network performance and internal representations.

Chat is not available.