A Robust Deep Attention Network to Noisy Labels in Semi-supervised Biomedical Segmentation
share
Learning-based methods suffer from limited clean annotations, especially for biomedical segmentation. For example, the noisy labels make model confused and the limited labels lead to an inadequate training, which are usually concomitant. In this paper, we propose a deep attention networks (DAN) that is more robust to noisy labels by eliminating the bad gradients caused by noisy labels, using attention modules. Especially, the strategy of multi-stage filtering is applied, because clear elimination in a certain layer is impossible. As the prior knowledge of noise distribution is usually unavailable, a two-stream network is developed to provide information from each other for attention modules to mine potential distribution of noisy gradients. The intuition is that a discussion of two students may find out mistakes taught by teacher. And we further analyse the infection processing of noisy labels and design three attention modules, according to different disturbance of noisy labels in different layers. Furthermore, a hierarchical distillation is developed to provide more reliable pseudo labels from unlabeld data, which further boosts the DAN. Combining our DAN and hierarchical distillation can significantly improve a model performance with deficient clean annotations. The experiments on both HVSMR 2016 and BRATS 2015 benchmarks demonstrate the effectiveness of our method.
READ FULL TEXT