Adapter-based Approaches to Knowledge-enhanced Language Models -- A Survey
Knowledge-enhanced language models (KELMs) have emerged as promising tools to bridge the gap between large-scale language models and domain-specific knowledge. KELMs can achieve higher factual accuracy and mitigate hallucinations by leveraging knowledge graphs (KGs). They are frequently combined with adapter modules to reduce the computational load and risk of catastrophic forgetting. In this paper, we conduct a systematic literature review (SLR) on adapter-based approaches to KELMs. We provide a structured overview of existing methodologies in the field through quantitative and qualitative analysis and explore the strengths and potential shortcomings of individual approaches. We show that general knowledge and domain-specific approaches have been frequently explored along with various adapter architectures and downstream tasks. We particularly focused on the popular biomedical domain, where we provided an insightful performance comparison of existing KELMs. We outline the main trends and propose promising future directions.
Code (0)
등록된 구현이 없습니다.
Tasks
General KnowledgeKnowledge GraphsSurveySystematic Literature ReviewMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Combining pre-trained language models and structured knowledge
In recent years, transformer-based language models have achieved state of the art performance in various NLP benchmarks. These models are able to extract mostly distributional information with some semantics from unstruc…
Knowledge GraphsKnowledge Enhanced Pretrained Language Models: A Compreshensive Survey
Pretrained Language Models (PLM) have established a new paradigm through learning informative contextualized representations on large-scale text corpus. This new paradigm has revolutionized the entire field of natural la…
SurveyContext-Aware Adapter Tuning for Few-Shot Relation Learning in Knowledge Graphs
Knowledge graphs (KGs) are instrumental in various real-world applications, yet they often suffer from incompleteness due to missing relations. To predict instances for novel relations with limited training examples, few…
Knowledge GraphsMeta-LearningRelationAuto-selected Knowledge Adapters for Lifelong Person Re-identification
Lifelong Person Re-Identification (LReID) extends traditional ReID by requiring systems to continually learn from non-overlapping datasets across different times and locations, adapting to new identities while preserving…
Lifelong learningPerson Re-IdentificationInjecting Domain-Specific Knowledge into Large Language Models: A Comprehensive Survey
Large Language Models (LLMs) have demonstrated remarkable success in various tasks such as natural language understanding, text summarization, and machine translation. However, their general-purpose nature often limits t…
Machine TranslationNatural Language UnderstandingText Summarization