First Story Detection (FSD) aims to identify the first story for an emerging event previously unreported, which is essential to practical applications in news analysis, intelligence gathering, and national security. Compared to information retrieval, text clustering, text classification, and other subject-based tasks, FSD is event-based and thus faces the challenging issues of multiple events on the same subject and the evolution of events. To tackle these challenges, several schemes for exploiting temporal information, named entity, and topic modeling, have been proposed for FSD. In this paper, we present a new term weighting scheme called LGT, which jointly models the Local element, Global element, and Topical association of each story. An unsupervised algorithm based on LGT is then devised and applied to FSD. We evaluate 4 feature reduction strategies and test our LGT scheme on an online model. Experiments show that our approach yields better results than existing baseline schemes on both retrospective and online FSD.
Bibliographical noteThis paper is an extension of our previous work. Compared to our previous work, we have added the following new contents in this paper: (1) the LGT is elaborated and compared theoretically in Section 3.2.1; (2) three existing strategies and a newly proposed nonparametric method of feature reduction are included in Section 3.2.2; (3) the experimental part is further extended by analyzing the performance with different topic numbers in Section 4.4.1; (4) four feature reduction methods are evaluated and compared on two datasets in Section 4.4.2; and (5) we have added more discussions on related studies in Section 2, and made many improvements on the introduction, method, experimental analysis, conclusion, and future research directions.
- Feature reduction
- First story detection
- Latent Dirichlet allocation