A Multi-Grained Perception Model for Sentiment Analysis with Perceived Contrastive Focal Loss

Multimodal sentiment analysis uses text, visual, and audio data to assess user sentiment, while both the discrimination power of modalities and sample distributions over categories remain imbalanced in practice. To address these challenges, we propose a Multi-grained Perception Model with Perceived Contrastive Focal loss, denoted MGSA1. More specifically, we design a Multi-grained Cross-modal Attention Perception (MCP) module, which employs coarse-grained and fine-grained cross-modal attention to deeply explore the complementary semantics between modalities, thereby modeling sentiment polarity and intensity by fusing text-video and text-audio data, respectively. Modeling sentiment polarity and intensity helps alleviate feature interference between modalities due to their differing discriminative power. Furthermore, the Perceived Contrastive Focal (PCF) loss is designed to address the challenge of unbalanced samples. We enhance the focal loss by incorporating inverse document frequency to dynamically weight samples within each class. Furthermore, information noise contrastive estimation is introduced to replace the class probability predictions in the enhanced focal loss, thereby more effective differentiation between positive and negative samples. Experiments on the MOSI and MOSEI datasets demonstrate that MGSA outperforms all baselines across a range of metrics.