Is Saliency Really Captured By Gradient?
Nehal Yasin · Jonathon Hare · Antonia Marcu
2024 Poster Session
in
Workshop: Scientific Methods for Understanding Neural Networks
in
Workshop: Scientific Methods for Understanding Neural Networks
Abstract
Numerous feature attribution (or saliency) measures have been proposed that utilise the gradients of the output with respect to features. Gradients in this setting unequivocally tell us about feature sensitivity by definition of the gradient, but do they really tell us about feature importance? We challenge the idea that sensitivity and importance are the same, and empirically show that gradients do not necessarily find important features that should be attributed to a models' prediction.
Chat is not available.
Successful Page Load