Personalizing Foundation Models for Cancer Imaging: A Study on Lymph Node Segmentation with SAM2 and MedSAM2

Foundation models such as SAM2 and MedSAM2 offer powerful zero-shot capabilities for segmentation tasks but require adaptation for clinically relevant, small-structure detection such as lymph nodes in abdominal CT scans. This study investigates the fine-tuning of these models using limited annotations under three training regimes—zero-shot, few-shot, and big-shot—across multiple loss functions tailored for class imbalance. Our experiments reveal that MedSAM2 achieved 83.5% Dice score, with its medical-specific pretraining and memory-augmented attention, delivers more stable and accurate performance, particularly with its custom loss combining Focal and Dice losses. In contrast, SAM2 offers faster training and competitive Dice scores, especially with focal-based losses, but exhibits greater variance and instability during convergence with other loss functions. Training-time comparisons and visual analyses further validate the robustness of MedSAM2 outputs across challenging cases. These findings highlight the potential of adapting foundation models with personalized, loss-aware strategies for small-structure segmentation, offering scalable solutions for real-world clinical imaging applications.