Multimodal fusion network with contrary latent topic memory for rumor detection

Jiaxin CHEN, Zekai WU, Zhenguo YANG, Haoran XIE, Fu Lee WANG, Wenyin LIU

Research output: Journal PublicationsJournal Article (refereed)peer-review

Abstract

Rumors can mislead readers and even have a negative impact on public events, especially multimodal rumors with text and images, which are easier to attract readers' attention. Most existing methods focus on capturing specific characteristics of rumor events and have difficulty in identifying unknown rumor events. In this paper, we propose a multimodal rumor detection network (termed as MRDN) for social rumor detection. MRDN combines the complementary information of text and images through the mechanism of multi-head self-attention fusion (MSF), which allocates weight to different modalities to carry out feature fusion from multiple perspectives. Furthermore, MRDN exploits contrary latent topic memory network (CLTM) to store semantic information about true and false patterns of rumors, which is useful for identifying upcoming new rumors. Extensive experiments conducted on three public datasets show that our multimodal rumor detection method outperforms the state-of-the-art approaches.
Original languageEnglish
Pages (from-to)104-113
Number of pages10
JournalIEEE Multimedia
Volume29
Issue number1
Early online date1 Feb 2022
DOIs
Publication statusPublished - Feb 2022

Bibliographical note

Publisher Copyright:
© 2022 IEEE.

Keywords

  • Data mining
  • Explosions
  • Feature extraction
  • Fuses
  • Semantics
  • Social networking (online)
  • Visualization

Fingerprint

Dive into the research topics of 'Multimodal fusion network with contrary latent topic memory for rumor detection'. Together they form a unique fingerprint.
  • Multimodal Fusion Network with Latent Topic Memory for Rumor Detection

    CHEN, J., WU, Z., YANG, Z., XIE, H., WANG, F. L. & LIU, W., 9 Jun 2021, 2021 IEEE International Conference on Multimedia and Expo, ICME 2021. IEEE Computer Society, 6 p. (Proceedings - IEEE International Conference on Multimedia and Expo).

    Research output: Book Chapters | Papers in Conference ProceedingsConference paper (refereed)Researchpeer-review

    4 Citations (Scopus)

Cite this