Few-Shot Named Entity Recognition via Meta-Learning

Jing LI, Billy CHIU, Shanshan FENG, Hao WANG

Research output: Journal PublicationsJournal Article (refereed)peer-review

76 Citations (Scopus)

Abstract

Few-shot learning under the N-way K-shot setting (i.e., K annotated samples for each of N classes) has been widely studied in relation extraction (e.g., FewRel) and image classification (e.g., Mini-ImageNet). Named entity recognition (NER) is typically framed as a sequence labeling problem where the entity classes are inherently entangled together because the entity number and classes in a sentence are not known in advance, leaving the N-way K-shot NER problem so far unexplored. In this paper, we first formally define a more suitable N-way K-shot setting for NER. Then we propose FewNER, a novel meta-learning approach for few-shot NER. FewNER separates the entire network into a task-independent part and a task-specific part. During training in FewNER, the task-independent part is meta-learned across multiple tasks and the task-specific part is learned for each individual task in a low-dimensional space. At test time, FewNER keeps the task-independent part fixed and adapts to a new task via gradient descent by updating only the task-specific part, resulting in it being less prone to overfitting and more computationally efficient. Compared with pre-Trained language models (e.g., BERT and ELMo) which obtain the transferability in an implicit manner (i.e., relying on large-scale corpora), FewNER explicitly optimizes the capability of 'learning to adapt quickly' through meta-learning. The results demonstrate that FewNER achieves state-of-The-Art performance against nine baseline methods by significant margins on three adaptation experiments (i.e., intra-domain cross-Type, cross-domain intra-Type and cross-domain cross-Type).

Original languageEnglish
Pages (from-to)4245-4256
Number of pages12
JournalIEEE Transactions on Knowledge and Data Engineering
Volume34
Issue number9
Early online date17 Nov 2020
DOIs
Publication statusPublished - Sept 2022
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 1989-2012 IEEE.

Keywords

  • Few-shot learning
  • Meta-learning
  • Natural language processing
  • Sequence labeling

Fingerprint

Dive into the research topics of 'Few-Shot Named Entity Recognition via Meta-Learning'. Together they form a unique fingerprint.

Cite this