Debugging and Explaining Metric Learning Approaches: An Influence Function Based Perspective

Ruofan Liu · Yun Lin · XIANGLIN YANG · Jin Song Dong

Keywords: [ noisy data ] [ influence function ] [ Metric Learning ]

[ Abstract ]
[ Poster [ OpenReview
Spotlight presentation: Lightning Talks 3B-3
Wed 7 Dec 10 a.m. PST — 10:15 a.m. PST


Deep metric learning (DML) learns a generalizable embedding space where the representations of semantically similar samples are closer. Despite achieving good performance, the state-of-the-art models still suffer from the generalization errors such as farther similar samples and closer dissimilar samples in the space. In this work, we design an empirical influence function (EIF), a debugging and explaining technique for the generalization errors of state-of-the-art metric learning models. EIF is designed to efficiently identify and quantify how a subset of training samples contributes to the generalization errors. Moreover, given a user-specific error, EIF can be used to relabel a potentially noisy training sample as mitigation. In our quantitative experiment, EIF outperforms the traditional baseline in identifying more relevant training samples with statistical significance and 33.5% less time. In the field study on well-known datasets such as CUB200, CARS196, and InShop, EIF identifies 4.4%, 6.6%, and 17.7% labelling mistakes, indicating the direction of the DML community to further improve the model performance. Our code is available at

Chat is not available.