Confabulation Instead of Abstention: How Medical Vision–Language Models Respond to Missing Visual Evidence

Medical vision--language models (VLMs) can produce fluent diagnostic descriptions even when the visual evidence supporting those descriptions is missing. We study this behavior using 100 dermoscopic images from ISIC 2016 and paired lesion-masked versions in which diagnostically relevant regions are removed. Three open-source VLMs are evaluated for evidence awareness, uncertainty change, causal attribution of uncertainty, and overconfidence. Models rarely recognize the loss of diagnostic evidence: the best model identified it in only 15% of masked responses. Biomedical fine-tuning did not improve this behavior and, in our experiments, reduced caution after evidence removal. Qualitative analysis further shows that models sometimes replace missing evidence with plausible but unsupported findings rather than abstaining. These results highlight confabulation under missing visual evidence as a distinct medical VLM failure mode and motivate broader perturbation-based evaluation across models and imaging modalities.