Rounding-Invariant Objectives: Is 1-Bit Enough?

Binary (1-bit) embeddings are treated as a lossy approximation to float: cheaper to store, faster to compare, worse at everything else. For structure-preserving data mining, we find this framing does not always hold. Sign-binarizing frozen embeddings from five open models — with no binarization-aware training and no algorithmic change beyond the distance matrix — leaves hierarchical agglomerative clustering at parity with float under complete linkage (median relative NMI gap 3.9% across 20 model–dataset pairs; binary is better in 8, and worse by at most 12.6%). Under average linkage the ordering inverts: float collapses on 10 of 12 modern-model/short-text pairs into a single cluster absorbing 94–99% of points, where binary on identical embeddings recovers balanced structure. Standard diagnostics do not separate the failing configurations from the working ones: on 20 Newsgroups, distance concentration is near-identical across models, and the collapse is abrupt rather than the classical chaining pattern; we point to the upper tail of the pairwise-distance distribution as the candidate mechanism. We argue the missing piece is a theory of which structure-preserving objectives are invariant to rounding — means break, extrema survive — and pose open questions in theory, algorithms and deployment: characterize that invariance, treat binary as default-eligible in clustering, and have open embedding models ship binary and mixed-precision representations as trained, parallel outputs. This matters most where memory rather than accuracy is the binding constraint, as in many resource-constrained Global South deployments.