Deductive Constraint Satisfaction vs. Prevalence Priors: Benchmarking LLM Logic in Clinical Diagnostics

Clinical LLMs encode substantial medical knowledge, but their reliability is less clear when explicit evidence supports a rare diagnosis while prevalence favors a common alternative. We introduce DiagRare, a controlled benchmark for separating deductive constraint satisfaction from associative pattern matching and quantifying how prevalence priors affect diagnostic ranking. DiagRare comprises an expert-curated ontology of 58 diseases (28 rare and 30 common), 358 clinical findings, and prevalence estimates spanning four orders of magnitude across six organ systems. From this ontology, we generate 696 logically consistent vignettes across three difficulty tiers by varying required findings, explicit exclusions, and controlled noise. We evaluate Gemini 3 Thinking, DeepSeek-V3.2, Claude Sonnet 4.6, GPT-5.4 Thinking, and GPT-5.3 Instant zero-shot, asking each model to rank the three most likely diagnoses from the same set of 58 candidates. Top-1 accuracy ranges from 31.6% to 100.0%. The weakest model exhibits prevalence-driven mode collapse, assigning 278 of its 476 errors to a single common diagnosis across 45 distinct target diseases and reducing diagnostic entropy to 3.33 bits. Stronger models fail more subtly: Claude Sonnet 4.6 includes the correct rare diagnosis in its Top-3 predictions in 99.7% of cases but ranks it first in only 86.3%, revealing a 13.4-point inductive downgrade. Increasing explicit negative constraints improves Claude’s accuracy from 84.5% to 97.0%, whereas adding positive findings reduces it from 97.1% to 85.7%. These results show that identifying the correct diagnosis and selecting it are distinct capabilities, and that Top-k accuracy can conceal prior-driven failures at the final decision stage.