Beyond 'This Looks Like That': Auditing Prototypes for Demographic Bias

Beyond 'This Looks Like That': Auditing Prototypes for Demographic Bias Prototypical-part networks (PPNs) (Chen et al., 2019) are a popular interpretable alternative to black-box image classifiers. Each prediction is grounded in similarity to learned prototype patches drawn from training images, producing explanations of the form "this looks like that." PPNs have been deployed in high-stakes domains including medical imaging (Barnett et al. 2021) and have inspired numerous architectural variants (Deformable Protopnet, ProtoTree). Their appeal rests on a simple promise: a prediction is justified by visual similarity to a specific, named, human-inspectable prototype, rather than by the opaque computations of a deep network. However, this promise contains a hidden risk. If the prototype layer learns from demographically imbalanced data, then "this looks like that" can quietly mean "this looks like a member of the majority demographic". The prediction is interpretable in the narrow sense; every decision is explained by a specific prototype, but the prototype itself may encode a demographic shortcut. Standard fairness metrics such as demographic parity gap detect this at the level of model outputs but cannot localize the bias to specific model components, making it difficult to understand or remediate. We propose an audit that operates on the prototype layer rather than the model's outputs: Do prototype-based models inherit demographic bias from their training data, and if so, can the prototype layer itself be audited for that bias? Specifically, we demonstrate a three-metric audit framework for demographic bias in prototype-based models: Prototype Demographic Unbalance Score identifies prototypes whose top-K nearest training images concentrate in a single demographic group; Prototype Activation Disparity measures whether different demographic groups activate different prototypes to reach the same correct prediction, using Jensen-Shannon divergence; Post-Suppression Accuracy Drop zeros each flagged prototype in turn and measures the per-group accuracy cost, providing causal evidence of demographic shortcuts. We applied the framework to ProtoPNet variants trained on CelebA's (Liu et al. 2015) smiling attribute, sweeping three CNN backbones and two activation functions (Cosine, L2). All six configurations achieved an AUROC of approx 0.98 yet exhibited demographic parity gaps of 0.27-0.29. At the prototype level, every flagged prototype in every configuration is dominated by young women, the majority demographic among smiling-positive training samples; men are depleted, and older women, despite the highest smiling base rate (0.69), are underrepresented in positive-class prototypes. The model amplified the training-set skew, raising young women's prototype representation from 55% in training to 82% in prototype dominance. When we suppressed flagged prototypes, older women lost the most accuracy (-9.7 pp for cosine DenseNet-161), confirming that the model classifies older women using prototype patterns learned from young women. The L2 model reversed this pattern, suggesting that activation function choice also shapes which group bears the cost. Because prototypes are discrete and named, the audit yields an actionable target list. Building on Rashomon-set methods for prototype models (Proto-RSet), we are developing an automated debiasing procedure that (a) scores each prototype by its contribution to the gap between the model's coefficient-weighted evidence base and true class demographics, (b) greedily removes prototypes while remaining within a user-set accuracy tolerance, and (c) resamples candidate prototypes from demographically under-drawn training images before refitting, enlarging the evidence pool rather than merely pruning it. Preliminary experiments confirm biased prototypes can be removed without accuracy loss; full ablations (selection vs. resampling vs. both), plus extension to dermatology (HAM10000, melanoma detection with sex/age groups), are in progress. Our framework identifies which prototypes encode demographic shortcuts before deployment, providing a concrete intervention point. The broader argument is that prototype-based interpretability does not entail fairness, but the implicit, named prototypes of these models enable diagnostic precision that black-box fairness audits cannot provide.