MCMT-GAN : Multi-Task Coherent Modality Transferable GAN for 3D Brain Image Synthesis

Yawen HUANG*, Feng ZHENG, Runmin CONG, Weilin HUANG, Matthew R. SCOTT, Ling SHAO

*Corresponding author for this work

Research output: Journal PublicationsJournal Article (refereed)peer-review

30 Citations (Scopus)


The ability to synthesize multi-modality data is highly desirable for many computer-aided medical applications, e.g. clinical diagnosis and neuroscience research, since rich imaging cohorts offer diverse and complementary information unraveling human tissues. However, collecting acquisitions can be limited by adversary factors such as patient discomfort, expensive cost and scanner unavailability. In this paper, we propose a multi-task coherent modality transferable GAN (MCMT-GAN) to address this issue for brain MRI synthesis in an unsupervised manner. Through combining the bidirectional adversarial loss, cycle-consistency loss, domain adapted loss and manifold regularization in a volumetric space, MCMT-GAN is robust for multi-modality brain image synthesis with visually high fidelity. In addition, we complement discriminators collaboratively working with segmentors which ensure the usefulness of our results to segmentation task. Experiments evaluated on various cross-modality synthesis show that our method produces visually impressive results with substitutability for clinical post-processing and also exceeds the state-of-the-art methods.

Original languageEnglish
Article number9152126
Pages (from-to)8187-8198
Number of pages12
JournalIEEE Transactions on Image Processing
Early online date29 Jul 2020
Publication statusPublished - 2020
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 1992-2012 IEEE.


  • anatomical structure
  • brain MRI
  • GANs
  • multi-modality
  • Synthesis


Dive into the research topics of 'MCMT-GAN : Multi-Task Coherent Modality Transferable GAN for 3D Brain Image Synthesis'. Together they form a unique fingerprint.

Cite this