Poster
in
Workshop: UniReps: Unifying Representations in Neural Models
On consequences of finetuning on data with highly discriminative features
Wojciech Masarczyk · Tomasz Trzcinski · Mateusz Ostaszewski
[
Abstract
]
[ Project Page ]
presentation:
UniReps: Unifying Representations in Neural Models
Fri 15 Dec 6:15 a.m. PST — 3:15 p.m. PST
[
OpenReview]
Fri 15 Dec 6:15 a.m. PST — 3:15 p.m. PST
Abstract:
In the era of transfer learning, training neural networks from scratch is becoming obsolete. Transfer learning leverages prior knowledge for new tasks, conserving computational resources. While its advantages are well-documented, we uncover a notable drawback: networks tend to prioritize basic data patterns, forsaking valuable pre-learned features. We term this behavior "feature erosion" and analyze its impact on network performance and internal representations.
Chat is not available.