SMI: Semantic Medical ID for Hierarchy-Aware Concept Representation

While biomedical language models (LMs) achieve strong performance on various clinical tasks, they often fail to capture the structured relationships among medical concepts. Because LMs rely on data-driven text learning and fail to learn meaningful patterns from long-tailed distributed medical concepts. We introduce Semantic Medical Identifier (SMI), a simple but effective representation framework that integrates expert-curated medical ontologies with LMs to generate hierarchy-aware concept embeddings. For each clinical code, SMI aggregates embeddings across hierarchical levels, leading to a representation that preserves hierarchical semantics from coarse to fine-grained clinical concepts. Our theoretical analysis shows that SMI yields a strictly higher signal-to-noise ratio than hierarchy-agnostic LM embeddings, particularly improving the representation of infrequent fine-grained concepts. Experimental results show that SMI learns interpretable geometry, improves concept relation discrimination and clinical prediction, and is robust to distribution shifts.