Predicting Species Splits: A Challenging Fine-Grained Benchmark for Category Discovery

Widely used category discovery benchmarks have become increasingly saturated and often fail to reflect real-world discovery settings, making long-term evaluation of new methods difficult. We introduce \textbf{PS-SPLIT}, a large-scale fine-grained benchmark for category discovery, motivated by taxonomic splitting: the real-world process by which what was once considered a single species is found to comprise multiple distinct ones. PS-SPLIT contains 79K globally distributed, high-quality, community-collected bird images spanning 771 species, derived from 257 recent real taxonomic splits, with associated geographic metadata and full taxonomic hierarchy. With categories arising from real taxonomic splits, visual differences can be very subtle, often requiring significant human expertise to tell apart, making PS-SPLIT a challenging benchmark which we hope will assist long term in fine-grained category discovery research. Based on taxonomic splitting, we introduce the task of \textbf{discovery by category splitting}, where the goal is to determine which known categories should be subdivided into finer-grained subcategories. We adapt four generalized category discovery methods to this setting and show that there is substantial room for future progress.