Multimodal Fusion Network with Latent Topic Memory for Rumor Detection

Jiaxin CHEN, Zekai WU, Zhenguo YANG, Haoran XIE, Fu Lee WANG, Wenyin LIU

Research output: Book Chapters | Papers in Conference ProceedingsConference paper (refereed)Researchpeer-review

4 Citations (Scopus)

Abstract

In this paper, we propose a multimodal fusion network (termed as MFN) to integrate the text and image data from social media for rumor detection. Given the multimodal features, MFN exploits self-attentive fusion (SAF) mechanism to conduct feature-level fusion by assigning corresponding weights to the complementary modalities. In particular, the textual features are combined with the fused features in a skip-connection manner, as textual features tend to be more distinguishable compared with visual features. Furthermore, MFN introduces latent topic memory (LTM) to store the semantic information about rumor and non-rumor events, benefiting to the identification of the upcoming posts. Extensive experiments conducted on two public datasets show that the proposed MFN outperforms the state-of-the-art approaches.
Original languageEnglish
Title of host publication2021 IEEE International Conference on Multimedia and Expo, ICME 2021
PublisherIEEE Computer Society
Number of pages6
ISBN (Electronic)9781665438643
DOIs
Publication statusPublished - 9 Jun 2021
Event2021 IEEE International Conference on Multimedia and Expo, ICME 2021 - Shenzhen, China
Duration: 5 Jul 20219 Jul 2021

Publication series

NameProceedings - IEEE International Conference on Multimedia and Expo
ISSN (Print)1945-7871
ISSN (Electronic)1945-788X

Conference

Conference2021 IEEE International Conference on Multimedia and Expo, ICME 2021
Country/TerritoryChina
CityShenzhen
Period5/07/219/07/21

Bibliographical note

Funding Information:
This work is supported by the National Natural Science Foundation of China (No.62076073), the Guangdong Basic and Applied Basic Research Foundation (No. 2020A1515010616), the Guangdong Innovative Research Team Program (No.2014ZT05G157), the Key-Area Research and Development Program of Guangdong Province (2019B010136001), and the Science and Technology Planning Project of Guangdong Province (LZC0023), a grant from the RGC of HKSAR, China (UGC/FDS16/E01/19), and the Faculty Research Fund (DB21A9), Lingnan University, Hong Kong.

Publisher Copyright:
© 2021 IEEE

Keywords

  • Multimodal fusion
  • Self-attentive
  • Rumor detection

Fingerprint

Dive into the research topics of 'Multimodal Fusion Network with Latent Topic Memory for Rumor Detection'. Together they form a unique fingerprint.

Cite this