Hybrid Diffusion Model for Breast Ultrasound Image Augmentation

We propose a hybrid diffusion-based augmentation framework to overcome the critical challenge of limited and imbalanced data in breast ultrasound (BUS) datasets. Unlike conventional augmentations, our approach captures ultrasound-specific features such as speckle noise by combining text-to-image generation with image-to-image (img2img) refinement as well as fine-tuning using LoRA and textual inversion (TI). Our method generated realistic, class-consistent images on an open-source Kaggle breast ultrasound image dataset (BUSI). Incorporating TI and img2img refinement on the Stable Diffusion v1.5 backbone reduced the Fréchet Inception Distance (FID) from 45.97 to 33.29, demonstrating a substantial gain in fidelity while maintaining comparable downstream classification performance. Overall, the proposed framework effectively mitigates the low-fidelity limitations of synthetic ultrasound images and enhances the quality of augmentation for robust diagnostic modeling.