A Patient-Level Multi-Magnification Hybrid CNN–VIT Fusion Model for Cervical Abnormality Detection

In recent years, cervical cancer has remained a significant global health concern, particularly in low- and middle-income countries (LMICs), where inadequate screening programs and a lack of on-ground pathologists contribute to late diagnosis, higher incidence, and mortality rates. Early, precise, and interpretable diagnostic techniques are therefore essential for improving patient outcomes. Colposcopy provides a visual means of identifying suspicious cervical abnormalities, and most examinations yield multiple magnified views of the same patient, yet automated approaches typically classify at the level of individual images. We investigate the use of multiple magnification views to jointly represent automated patient-level identification of an abnormal cervix, using a dataset gathered from an LMIC setting in Nigeria comprising 3,356 images from 332 women, acquired at four magnification levels before and after acetic acid application, with expert-assigned Normal and Abnormal colposcopy labels characterized by visual features including acetowhite changes, punctation, and mosaicism. While hybrid convolutional neural network (CNN) and Vision Transformer (ViT) approaches have previously been explored for cervical image analysis, to the best of our knowledge, this is the first reported investigation of a patient-level multi-magnification hybrid CNN–ViT fusion on this dataset, aimed at resource-constrained settings. Our framework models each pre- and post-acetic magnification through complementary local and global visual pathways. Controlled preprocessing experiments showed that direct whole-image resizing preserved stronger discrimination than letterboxing or CLAHE, while the cervical region of interest (ROI) remained useful as a complementary local representation. This motivated a dual-view architecture in which a CNN processes the ROI to capture localized morphology and texture. In contrast, the corresponding whole image is processed by a ViT to preserve global cervical context. Our study addresses three questions. First, how much model capacity is required? For each magnification, we compare full-capacity and lightweight hybrid models under an identical patient-level protocol. Second, how should local and global representations interact? Using the selected lightweight encoders, we compare bidirectional and unidirectional CNN–ViT fusion, keeping the multi-magnification patient aggregation fixed. Third, are generic natural-image representations sufficient? Motivated by the cohort's limited size, we investigate the self-supervised adaptation of training-set colposcopy images to learn cervical-domain representations prior to supervised classification. We additionally applied Grad-CAM to examine the regions the model relied on for its decisions. Our findings showed that increasing model capacity did not improve discrimination, redirecting our investigation toward fusion directionality and representation quality rather than further model scaling. Bidirectional CNN–ViT fusion outperformed unidirectional fusion, indicating that mutual local–global interactions matter more than raw model capacity in identifying cervical abnormalities. In contrast, self-supervised adaptation of colposcopy images to learn cervical-domain representations yielded little additional gain, suggesting that architectural interaction between local and global pathways, rather than domain-specific pretraining alone, is the primary lever for resource-efficient, patient-level, multi-view modelling of cervical abnormalities in resource-constrained settings. References [1] World Health Organization. Cervical cancer. WHO, 2026. [2] Ogunlaja, A.O., Adeleke, O.T., Ughonu, P.C., et al. A multi-magnification digital colposcopy image datasets for cervical cancer screening in Nigeria. Data in Brief 65, 112633 (2026). [3] Sarhangi, H. A., Beigifard, D., Farmani, E., & Bolhasani, H. (2024). Deep learning techniques for cervical cancer diagnosis based on pathology and colposcopy images. Informatics in Medicine Unlocked, 47, 101503.