From BERT to T5: A Study of Named Entity Recognition
Named entity recognition (NER) has been one of the essential preliminary steps in modern NLP applications. This report focuses on implementing the NER task on finetuning two pretrained models: (i) an encoder-only model (BERT) with a simple classification head, and (ii) a sequence-to-sequence model (T5) with few-shot prompts. Under the original 7-class tag and 3-class simplified tag schemes, BERT is applied a weighted cross-entropy for training loss, and T5 is fine-tuned with two validation strategies. It also conducted an ablation study with different hyperparameters. Moreover, the related analysis provides valuable insights into common errors in BERT and the two models' performance. Based on a bunch of performance metrics, this report aims to compare the above two architectures and explore their abilities in the sequence labelling task, laying the groundwork for further practical use cases.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
RoBERTa-based Traditional Chinese Medicine Named Entity Recognition Model
In this study, a named entity recognition was constructed and applied to the identification of Chinese medicine names and disease names. The results can be further used in a human-machine dialogue system to provide peopl…
Articlesnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+1Chemical Identification and Indexing in PubMed Articles via BERT and Text-to-Text Approaches
The Biocreative VII Track-2 challenge consists of named entity recognition, entity-linking (or entity-normalization), and topic indexing tasks -- with entities and topics limited to chemicals for this challenge. Named en…
ArticlesChemical IndexingEntity LinkingMetric Learning+5Code and Named Entity Recognition in StackOverflow
There is an increasing interest in studying natural language and computer code together, as large corpora of programming texts become readily available on the Internet. For example, StackOverflow currently has over 15 mi…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NERWCL-BBCD: A Contrastive Learning and Knowledge Graph Approach to Named Entity Recognition
Named Entity Recognition task is one of the core tasks of information extraction. Word ambiguity and word abbreviation are important reasons for the low recognition rate of named entities. In this paper, we propose a nov…
Contrastive LearningKnowledge Graphsnamed-entity-recognitionNamed Entity Recognition+2Evaluating Pretrained Transformer-based Models on the Task of Fine-Grained Named Entity Recognition
Named Entity Recognition (NER) is a fundamental Natural Language Processing (NLP) task and has remained an active research field. In recent years, transformer models and more specifically the BERT model developed at Goog…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NER