Slot Attention and K-means

Slot attention was introduced by Locatello et al. (2020) as a variant of the attention algorithm. It was used to perform unsupervised image segmentation in an object centric way. In this paper, we theoretically study a simplification of slot attention under an interacting particle system perspective and connect it to K-means. Our goal is to understand how the isolated competitive assignment dynamics of slot attention perform clustering. Furthermore, we show experimentally how a parameter free variant performs similarly to the full slot attention model on image segmentation of the PascalVOC dataset using DINO features.