Timezone: »

Disentangling Human Error from Ground Truth in Segmentation of Medical Images
Le Zhang · Ryutaro Tanno · Mou-Cheng Xu · Chen Jin · Joseph Jacob · Olga Cicarrelli · Frederik Barkhof · Daniel Alexander

Thu Dec 10 09:00 PM -- 11:00 PM (PST) @ Poster Session 6 #1844

Recent years have seen increasing use of supervised learning methods for segmentation tasks. However, the predictive performance of these algorithms depends on the quality of labels. This problem is particularly pertinent in the medical image domain, where both the annotation cost and inter-observer variability are high. In a typical label acquisition process, different human experts provide their estimates of the ``true'' segmentation labels under the influence of their own biases and competence levels. Treating these noisy labels blindly as the ground truth limits the performance that automatic segmentation algorithms can achieve. In this work, we present a method for jointly learning, from purely noisy observations alone, the reliability of individual annotators and the true segmentation label distributions, using two coupled CNNs. The separation of the two is achieved by encouraging the estimated annotators to be maximally unreliable while achieving high fidelity with the noisy training data. We first define a toy segmentation dataset based on MNIST and study the properties of the proposed algorithm. We then demonstrate the utility of the method on three public medical imaging segmentation datasets with simulated (when necessary) and real diverse annotations: 1) MSLSC (multiple-sclerosis lesions); 2) BraTS (brain tumours); 3) LIDC-IDRI (lung abnormalities). In all cases, our method outperforms competing methods and relevant baselines particularly in cases where the number of annotations is small and the amount of disagreement is large. The experiments also show strong ability to capture the complex spatial characteristics of annotators' mistakes. Our code is available at \url{https://github.com/moucheng2017/LearnNoisyLabelsMedicalImages}.

Author Information

Le Zhang (University College London)
Ryutaro Tanno (Microsoft Research)

Senior Researcher at Microsoft Research Cambridge, UK PhD in Machine Learning

Mou-Cheng Xu (University College London)

I am a PhD student working at The Centre of Medical Image Computing, University College London. My project is applying machine learning on a large scale CT data set from a national lung screening project to study lung fibrosis diseases. I am interested in AI for healthcare, learning with imperfect data, computer vision.

Chen Jin (University College London)
Joseph Jacob (University College London)
Olga Cicarrelli (Queen Square Multiple Sclerosis Centre)
Frederik Barkhof (University College London)
Daniel Alexander (University College London)

More from the Same Authors