Breast Cancer Classification through Meta-Learning on Multimodal MRIs

This paper proposes a multimodal neural network AI model for gauging the metastatic load of axillary lymph nodes in the breast. The model utilizes three modalities of images, namely dynamic contrast enhancement (DCE), T2-weighted (T2W), and diffusion-weighted imaging (DWI), from breast magnetic resonance imaging (MRI) and axillary lymph node MRI. Features are extracted by a feature extractor (composed of conv1 and layer1 of ResNet and Wavelet transform convolution model) that is pretrained on a large breast cancer MRI dataset based on the Model-Agnostic Meta-Learning (MAML) algorithm. The features of the same modality from breast MRI and axillary lymph node MRI are concatenated and then input into the multimodal MulT model for classifications. The experimental results show that the addition of meta-learning and the involvement of multimodal MRI (rather than just uni-model MRI) significantly improve the classification, with the area under the ROC curve (AUC) reaching 0.84. The model performs well in judging the metastatic load of axillary lymph nodes in the breast and is expected to contribute to clinical diagnosis and treatments (both invasive and non-invasive).