Reconciliation of Pre-trained Models and Prototypical Neural Networks in Few-shot Named Entity Recognition

2022-11-07 · Youcheng Huang, Wenqiang Lei, Jie Fu, Jiancheng Lv

Incorporating large-scale pre-trained models with the prototypical neural networks is a de-facto paradigm in few-shot named entity recognition. Existing methods, unfortunately, are not aware of the fact that embeddings from pre-trained models contain a prominently large amount of information regarding word frequencies, biasing prototypical neural networks against learning word entities. This discrepancy constrains the two models' synergy. Thus, we propose a one-line-code normalization method to reconcile such a mismatch with empirical and theoretical grounds. Our experiments based on nine benchmark datasets show the superiority of our method over the counterpart models and are comparable to the state-of-the-art methods. In addition to the model enhancement, our work also provides an analytical viewpoint for addressing the general problems in few-shot name entity recognition or other tasks that rely on pre-trained models or prototypical neural networks.

📄 PDF Abstract BibTeX arXiv:2211.03270

Code (1)

hamlaertes/emnlp_2022_reconciliation 공식 구현 pytorch

Tasks

named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)

Methods 이 논문이 사용한 방법론

AWARE We propose to theoretically and empirically examine the effect of incorporating weighting schemes into walk-aggregating GNNs. To this end, we propose a simple, interpretable, and…

Label Embedding for Zero-shot Fine-grained Named Entity Typing

2016-12-01 · COLING 2016 12 · Yukun Ma, Erik Cambria, Sa Gao

Named entity typing is the task of detecting the types of a named entity in context. For instance, given {``}Eric is giving a presentation{''}, our goal is to infer that {`}Eric{'} is a speaker or a presenter and a perso…

Entity Linking Entity Typing Named Entity Recognition (NER)Question Answering+2

Few-shot Named Entity Recognition with Joint Token and Sentence Awareness

2021-11-16 · ACL ARR November 2021 11 · Anonymous

Few-shot learning has been proposed and rapidly emerging as a viable means for completing various tasks. Recently, few-shot models have been used for Named Entity Recognition (NER). Prototypical network shows high effici…

Few-Shot Learning Few-shot NER Named Entity Recognition Named Entity Recognition (NER)+3

Few-shot classification in Named Entity Recognition Task

2018-12-14 · Alexander Fritzler, Varvara Logacheva, Maksim Kretov

For many natural language processing (NLP) tasks the amount of annotated data is limited. This urges a need to apply semi-supervised learning techniques, such as transfer learning or meta-learning. In this work we tackle…

Classification General Classification Meta-Learning Metric Learning+6

Learning from Miscellaneous Other-Class Words for Few-shot Named Entity Recognition

2021-06-29 · ACL 2021 5 · Meihan Tong, Shuai Wang, Bin Xu, Yixin Cao 외

Few-shot Named Entity Recognition (NER) exploits only a handful of annotations to identify and classify named entity mentions. Prototypical network shows superior performance on few-shot NER. However, existing prototypic…

Few-shot NER Miscellaneous Named Entity Recognition Named Entity Recognition (NER)

SpanProto: A Two-stage Span-based Prototypical Network for Few-shot Named Entity Recognition

2022-10-17 · Jianing Wang, Chengcheng Han, Chengyu Wang, Chuanqi Tan 외

Few-shot Named Entity Recognition (NER) aims to identify named entities with very little annotated data. Previous methods solve this problem based on token-wise classification, which ignores the information of entity bou…

Few-shot NER Named Entity Recognition Named Entity Recognition (NER)