Merging Statistical Feature via Adaptive Gate for Improved Text Classification

Xianming LI, Zongxi LI*, Haoran XIE, Qing LI

*Corresponding author for this work

Research output: Book Chapters | Papers in Conference ProceedingsConference paper (refereed)Referred Conference Paperpeer-review

11 Citations (Scopus)

Abstract

Currently, text classification studies mainly focus on training classifiers by using textual input only, or enhancing semantic features by introducing external knowledge (e.g., hand-craft lexicons and domain knowledge). In contrast, some intrinsic statistical features of the corpus, like word frequency and distribution over labels, are not well exploited. Compared with external knowledge, the statistical features are deterministic and naturally compatible with corresponding tasks. In this paper, we propose an Adaptive Gate Network (AGN) to consolidate semantic representation with statistical features selectively. In particular, AGN encodes statistical features through a variational component and merges information via a well-designed valve mechanism. The valve adapts the information flow into the classifier according to the confidence of semantic features in decision making, which can facilitate training a robust classifier and can address the overfitting caused by using statistical features. Extensive experiments on datasets of various scales show that, by incorporating statistical information, AGN can improve the classification performance of CNN, RNN, Transformer, and Bert based models effectively. The experiments also indicate the robustness of AGN against adversarial attacks of manipulating statistical information.
Original languageEnglish
Title of host publication35th AAAI Conference on Artificial Intelligence, AAAI 2021
PublisherAssociation for the Advancement of Artificial Intelligence
Pages13288-13296
Number of pages9
ISBN (Electronic)9781713835974
Publication statusPublished - 18 May 2021
Event35th AAAI Conference on Artificial Intelligence, AAAI 2021 - Online, Virtual, Online
Duration: 2 Feb 20219 Feb 2021
https://ojs.aaai.org/index.php/AAAI/issue/view/399

Publication series

Name35th AAAI Conference on Artificial Intelligence, AAAI 2021
Volume15

Conference

Conference35th AAAI Conference on Artificial Intelligence, AAAI 2021
CityVirtual, Online
Period2/02/219/02/21
Internet address

Bibliographical note

Funding Information:
Xianming Li’s work has been supported by Ant Group. Zongxi Li’s work has been supported by City University of Hong Kong. Haoran Xie’s work has been supported by the Faculty Research Fund (102041) and the Lam Woo Research Fund (LWI20011) at Lingnan University, Hong Kong. Qing Li’s work has been supported by a general research fund from the Hong Kong Research Grants Council (project number: PolyU 112114/17E). We thank the anonymous reviewers for their careful reading of our manuscript and their insightful comments and suggestions.

Funding Information:
Xianming Li's work has been supported by Ant Group. Zongxi Li's work has been supported by City University of Hong Kong. Haoran Xie's work has been supported by the Faculty Research Fund (102041) and the LamWoo Research Fund (LWI20011) at Lingnan University, Hong Kong. Qing Li's work has been supported by a general research fund from the Hong Kong Research Grants Council (project number: PolyU 112114/17E). We thank the anonymous reviewers for their careful reading of our manuscript and their insightful comments and suggestions.

Publisher Copyright:
Copyright © 2021, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

Fingerprint

Dive into the research topics of 'Merging Statistical Feature via Adaptive Gate for Improved Text Classification'. Together they form a unique fingerprint.

Cite this