Contrastive learning for hierarchical topic modeling

Pengbo MAO, Hegang CHEN, Yanghui RAO*, Haoran XIE, Fu lee WANG

*Corresponding author for this work

Research output: Journal PublicationsJournal Article (refereed)peer-review

Abstract

Topic models have been widely used in automatic topic discovery from text corpora, for which, the external linguistic knowledge contained in Pre-trained Word Embeddings (PWEs) is valuable. However, the existing Neural Topic Models (NTMs), particularly Variational Auto-Encoder (VAE)-based NTMs, suffer from incorporating such external linguistic knowledge, and lacking of both accurate and efficient inference methods for approximating the intractable posterior. Furthermore, most existing topic models learn topics with a flat structure or organize them into a tree with only one root node. To tackle these limitations, we propose a new framework called as Contrastive Learning for Hierarchical Topic Modeling (CLHTM), which can efficiently mine hierarchical topics based on inputs of PWEs and Bag-of-Words (BoW). Experiments show that our model can automatically mine hierarchical topic structures, and have a better performance than the baseline models in terms of topic hierarchical rationality and flexibility.
Original languageEnglish
Article number100058
JournalNatural Language Processing Journal
Volume6
Early online date3 Feb 2024
DOIs
Publication statusPublished - Mar 2024

Bibliographical note

The work of Yanghui Rao was supported in part by the National Natural Science Foundation of China (62372483). The work of Haoran Xie was supported in part by Lam Woo Research Fund (LWP20019) and Faculty Research Grants (DB24A4 and DB23B2) of Lingnan University, Hong Kong. The work of Fu Lee Wang was supported in part by a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (UGC/FDS16/E01/19).

Keywords

  • Hierarchical topic modeling
  • Contrastive learning
  • Neural variational inference

Fingerprint

Dive into the research topics of 'Contrastive learning for hierarchical topic modeling'. Together they form a unique fingerprint.

Cite this