Tail Wags the Model: Generalization and Membership Privacy Trade-offs of Sharpness-Aware Minimization

Sharpness-Aware Minimization (SAM) has been reported to improve generalization while leaking more membership privacy than SGD. In this paper, we aim to theoretically explain this phenomenon under long-tailed distributions. In long-tailed settings, typical examples are explained by dominant majority features, whereas atypical tail examples depend on weaker minority features. We theoretically show that SAM moves farther than gradient descent along a minority-preserving trajectory under an overparameterized model. This bias improves the embedding-space separation between typical and atypical subclasses, reducing their representational entanglement. However, because the minority cluster is learned from a few atypical training examples, stronger minority alignment can also amplify finite-sample information from those examples. Consequently, atypical members become more aligned with the empirical atypical prototype than atypical nonmembers, creating a membership signal that can be exploited by the attacker. Finally, we validate our findings empirically. Our analysis explains the generalization--privacy tension of SAM through a single mechanism: in long-tailed learning, the tail wags the model.