Skip to yearly menu bar Skip to main content

Workshop: Medical Imaging Meets NeurIPS

Attention Transfer Outperforms Transfer Learning in Medical Image Disease Classifiers

Sina Akbarian


Convolutional neural networks (CNN) are widely used in medical images diagnostic. However, training the CNNs is prohibitive in a low-data environment. In this study, for the low-data medical image domain, we propose a novel knowledge transfer approach to facilitate the training of CNNs. Our approach adopts the attention transfer framework to transfer knowledge from a carefully pre-trained CNN teacher to a student CNN. The performance of the CNN models is then evaluated on three medical image datasets including Diabetic Retinopathy, CheXpert, and ChestX-ray8. We compare our results with the well-known and widely used transfer learning approach. We show that the teacher-student (Attention transfer) framework not only outperforms transfer learning, in both in-domain and cross-domain knowledge transfer but also behave as a regularizer.

Chat is not available.