Towards Characterizing Scientific Image Utility and Upgradability

Scientific images function as critical evidence in research communication, yet their integrity faces unprecedented threats from AI-generated content that introduces subtle but consequential errors. Existing evaluation paradigms prove inadequate: image quality assessment poorly correlate with scientific validity, while language models lack domain-specific verification capabilities. To address this gap, we propose the $\textbf{S}$cientific $\textbf{I}$mage $\textbf{U}$tility and $\textbf{U}$pgradability $\textbf{A}$ssessment ($\textbf{SIU$^2$A}$) framework, which introduces two complementary dimensions for scientific image evaluation. $\textbf{Utility}$ encompasses $\textit{error detection}$ (identifying scientific inaccuracies) and $\textit{correction feasibility}$ (assessing whether errors can be reliably repaired). $\textbf{Upgradability}$ measures the quality of correction We categorize scientific image corruption into four fundamental types: Detail Distortion, Incompleteness, False Content, and Entity Confusion. Based on this taxonomy, we construct SIU$^2$A-Benchmark, a comprehensive dataset featuring expert annotations for both error identification and repair. The framework implements a unified two-stage evaluation protocol: the first stage evaluates error detection and correction instruction generation (Utility), while the second stage assesses the effectiveness of actual corrections (Upgradability). Experiments reveal that current multimodal systems exhibit significant limitations in both scientific error assessment and faithful correction, exposing a fundamental gap between visual perception and scientific usability. Our findings establish that diagnostic quality fundamentally constrains restoration performance, highlighting the critical need for robust error perception mechanisms in scientific multimodal AI.