Cross-modal Attention Network with Orthogonal Latent Memory for Rumor Detection

Zekai WU, Jiaxin CHEN, Zhenguo YANG*, Haoran XIE, Fu Lee WANG, Wenyin LIU*

*Corresponding author for this work

Research output: Book Chapters | Papers in Conference ProceedingsConference paper (refereed)Researchpeer-review

Abstract

In this paper, we design a cross-modal attention fusion network with orthogonal latent memory (CALM) to fuse multi-modal social media data for rumor detection. Given multimodal content features extracted from text and images, we devise a cross-modal attention fusion (CAF) mechanism to extract critical information underlying the modalities by intra-modality attention, and model the underlying relations among the modalities by inter-modality attention. In terms of the text, the natural sequential characteristics are critical to semantic understanding, while existing sequence models suffer from losing the information conveyed by the former words. To this end, we propose a Bi-GRU with orthogonal latent memory to extract the sequential features from the text, where the memory captures independent patterns. The fused content features and the sequential features can be used for rumor detection seamlessly. Extensive experiments conducted on two real-world datasets show the outperformance of the proposed CALM. (e.g., F1 -score is improved from 0.823 to 0.846 on Weibo dataset).
Original languageEnglish
Title of host publicationWeb Information Systems Engineering – WISE 2021
Subtitle of host publication22nd International Conference on Web Information Systems Engineering, WISE 2021, Melbourne, VIC, Australia, October 26–29, 2021, Proceedings, Part I
EditorsWenjie ZHANG, Lei ZOU, Zakaria MAAMAR, Lu CHEN
PublisherSpringer, Cham
Pages527-541
Number of pages15
Edition1
ISBN (Electronic)9783030908881
ISBN (Print)9783030908874
DOIs
Publication statusPublished - Jan 2022
Event22nd International Conference on Web Information Systems Engineering - Melbourne, Australia
Duration: 26 Oct 202129 Oct 2021
http://www.wise-conferences.org/2021/

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13080 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference22nd International Conference on Web Information Systems Engineering
Abbreviated titleWISE 2021
Country/TerritoryAustralia
CityMelbourne
Period26/10/2129/10/21
Internet address

Bibliographical note

Funding Information:
Acknowledgement. This work is supported by the National Natural Science Foundation of China (No. 62076073), the Guangdong Basic and Applied Basic Research Foundation (No. 2020A1515010616), Science and Technology Program of Guangzhou (No. 202102020524), the Guangdong Innovative Research Team Program (No.2014Z T05G157), HKIBS Research Program Grant Application (HCRG-201-002) and the Faculty Research Grant (DB21B6) of Lingnan University, Hong Kong.

Publisher Copyright:
© 2021, Springer Nature Switzerland AG.

Keywords

  • Multi-modal
  • Rumor detection
  • Social media

Fingerprint

Dive into the research topics of 'Cross-modal Attention Network with Orthogonal Latent Memory for Rumor Detection'. Together they form a unique fingerprint.

Cite this