Bumblebee: Foundational Model for Particle Physics Discovery

Bumblebee is a foundational model for particle physics discovery, inspired by BERT. By removing positional encodings and embedding particle 4-vectors, Bumblebee captures both generator- and reconstruction-level information while ensuring sequence-order invariance. Pre-trained on a masked task, it improves top quark reconstruction resolution by 10-20\% and excels in downstream tasks, including toponium discrimination (AUROC 0.90) and initial state classification (AUROC 0.70). Bumblebee’s flexibility makes it well-suited for a wide range of particle physics applications, especially the discovery of new particles and possibly fast detector simulation.