Multimodal fusion network with contrary latent topic memory for rumor detection

Rumors can mislead readers and even have a negative impact on public events, especially multimodal rumors with text and images, which are easier to attract readers' attention. Most existing methods focus on capturing specific characteristics of rumor events and have difficulty in identifying unknown rumor events. In this paper, we propose a multimodal rumor detection network (termed as MRDN) for social rumor detection. MRDN combines the complementary information of text and images through the mechanism of multi-head self-attention fusion (MSF), which allocates weight to different modalities to carry out feature fusion from multiple perspectives. Furthermore, MRDN exploits contrary latent topic memory network (CLTM) to store semantic information about true and false patterns of rumors, which is useful for identifying upcoming new rumors. Extensive experiments conducted on three public datasets show that our multimodal rumor detection method outperforms the state-of-the-art approaches.