KARL-Trans-NER: Knowledge Aware Representation Learning for Named Entity Recognition using Transformers
The inception of modeling contextual information using models such as BERT, ELMo, and Flair has significantly improved representation learning for words. It has also given SOTA results in almost every NLP task - Machine Translation, Text Summarization and Named Entity Recognition, to name a few. In this work, in addition to using these dominant context-aware representations, we propose a Knowledge Aware Representation Learning (KARL) Network for Named Entity Recognition (NER). We discuss the challenges of using existing methods in incorporating world knowledge for NER and show how our proposed methods could be leveraged to overcome those challenges. KARL is based on a Transformer Encoder that utilizes large knowledge bases represented as fact triplets, converts them to a graph context, and extracts essential entity information residing inside to generate contextualized triplet representation for feature augmentation. Experimental results show that the augmentation done using KARL can considerably boost the performance of our NER system and achieve significantly better results than existing approaches in the literature on three publicly available NER datasets, namely CoNLL 2003, CoNLL++, and OntoNotes v5. We also observe better generalization and application to a real-world setting from KARL on unseen entities.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine Translationnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NERRepresentation LearningText SummarizationTripletWorld KnowledgeMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
KARL: Knowledge-Aware Retrieval and Representations aid Retention and Learning in Students
Flashcard schedulers rely on 1) student models to predict the flashcards a student knows; and 2) teaching policies to pick which cards to show next via these predictions. Prior student models, however, just use study dat…
Knowledge TracingRetrievalSchedulingKARL: Mitigating Hallucinations in LLMs via Knowledge-Boundary-Aware Reinforcement Learning
Enabling large language models (LLMs) to appropriately abstain from answering questions beyond their knowledge is crucial for mitigating hallucinations. While existing reinforcement learning methods foster autonomous abs…
Reinforcement LearningKARL: Knowledge-Aware Reasoning Memory Modeling with Reinforcement Learning of Vector Space
Founded in Atkinson-Shiffrin Memory Model's three-stage theory, the cognitive process of answering a question with stored knowledge can be seen as a reasoning process that goes from the external sensory memory via short-…
DecoderLanguage ModelingLanguage ModellingQuestion Answering+2Monotonic Representation of Numeric Properties in Language Models
Language models (LMs) can express factual knowledge involving numeric properties such as Karl Popper was born in 1902. However, how this information is encoded in the model's internal representations is not understood we…
Single-pass Adaptive Image Tokenization for Minimum Program Search
According to Algorithmic Information Theory (AIT) -- Intelligent representations compress data into the shortest possible program that can reconstruct its content, exhibiting low Kolmogorov Complexity (KC). In contrast, …
Representation LearningReinforcement Learning