Timezone: »

 
Poster
Class-Aware Adversarial Transformers for Medical Image Segmentation
Chenyu You · Ruihan Zhao · Fenglin Liu · Siyuan Dong · Sandeep Chinchali · Ufuk Topcu · Lawrence Staib · James Duncan

Wed Nov 30 02:00 PM -- 04:00 PM (PST) @ Hall J #137
in Poster Session 4 »

Transformers have made remarkable progress towards modeling long-range dependencies within the medical image analysis domain. However, current transformer-based models suffer from several disadvantages: (1) existing methods fail to capture the important features of the images due to the naive tokenization scheme; (2) the models suffer from information loss because they only consider single-scale feature representations; and (3) the segmentation label maps generated by the models are not accurate enough without considering rich semantic contexts and anatomical textures. In this work, we present CASTformer, a novel type of adversarial transformers, for 2D medical image segmentation. First, we take advantage of the pyramid structure to construct multi-scale representations and handle multi-scale variations. We then design a novel class-aware transformer module to better learn the discriminative regions of objects with semantic structures. Lastly, we utilize an adversarial training strategy that boosts segmentation accuracy and correspondingly allows a transformer-based discriminator to capture high-level semantically correlated contents and low-level anatomical features. Our experiments demonstrate that CASTformer dramatically outperforms previous state-of-the-art transformer-based approaches on three benchmarks, obtaining 2.54%-5.88% absolute improvements in Dice over previous models. Further qualitative experiments provide a more detailed picture of the model’s inner workings, shed light on the challenges in improved transparency, and demonstrate that transfer learning can greatly improve performance and reduce the size of medical image datasets in training, making CASTformer a strong starting point for downstream medical image analysis tasks.

Keywords: Generative Adversarial Network · Vision transformer · Medical Image Segmentation

Author Information

Chenyu You (Yale University)

Chenyu You is a Ph.D. student in the Department of Electrical Engineering, at Yale University, working with Professor James Duncan. He obtained his master degree in Electrical Engineering from Stanford University, specializing in Artificial Intelligence (AI) Prior to that, he received his bachelor degree (with highest honors) in Electrical Engineering and Mathematics from Rensselaer Polytechnic Institute (RPI). He is broadly interested in the fields of machine learning, computer/medical vision, natural language processing, signal processing, optimization, and interdisciplinary applications.

Ruihan Zhao (UT Austin)
Fenglin Liu (University of Oxford)
Siyuan Dong (Yale University)
Sandeep Chinchali (University of Texas, Austin)
Ufuk Topcu (The University of Texas at Austin)
Lawrence Staib (Yale)
James Duncan (Yale University)

More from the Same Authors