Large Language Model Enhanced Knowledge Representation Learning: A Survey

2024-07-01 · Xin Wang, Zirui Chen, Haofen Wang, Leong Hou U, Zhao Li, Wenbin Guo

The integration of Large Language Models (LLM) with Knowledge Representation Learning (KRL) signifies a significant advancement in the field of artificial intelligence (AI), enhancing the ability to capture and utilize both structure and textual information. Despite the increasing research on enhancing KRL with LLMs, a thorough survey that analyse processes of these enhanced models is conspicuously absent. Our survey addresses this by categorizing these models based on three distinct Transformer architectures, and by analyzing experimental data from various KRL downstream tasks to evaluate the strengths and weaknesses of each approach. Finally, we identify and explore potential future research directions in this emerging yet underexplored domain.

📄 PDF Abstract BibTeX arXiv:2407.00936

Code (0)

등록된 구현이 없습니다.

Tasks

Language ModelingLanguage ModellingLarge Language ModelRepresentation LearningSurvey

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Multi-Head Attention 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Position-Wise Feed-Forward Layer 설명 없음

A Survey of Knowledge Enhanced Pre-trained Models

2021-10-01 · Jian Yang, Xinyu Hu, Gang Xiao, Yulong Shen

Pre-trained language models learn informative word representations on a large-scale text corpus through self-supervised learning, which has achieved promising performance in fields of natural language processing (NLP) af…

Logical Reasoning Representation Learning Self-Supervised Learning Survey

Knowledge Enhanced Pretrained Language Models: A Compreshensive Survey

2021-10-16 · Xiaokai Wei, Shen Wang, Dejiao Zhang, Parminder Bhatia 외

Pretrained Language Models (PLM) have established a new paradigm through learning informative contextualized representations on large-scale text corpus. This new paradigm has revolutionized the entire field of natural la…

Survey

A Survey of Knowledge-Enhanced Text Generation

2020-10-09 · Wenhao Yu, Chenguang Zhu, Zaitang Li, Zhiting Hu 외

The goal of text generation is to make machines express in human language. It is one of the most important yet challenging tasks in natural language processing (NLP). Since 2014, various neural encoder-decoder models pio…

Decoder Survey Text Generation

Knowledge Integration Strategies in Autonomous Vehicle Prediction and Planning: A Comprehensive Survey

2025-02-13 · Kumar Manas, Adrian Paschke

This comprehensive survey examines the integration of knowledge-based approaches into autonomous driving systems, with a focus on trajectory prediction and planning. We systematically review methodologies for incorporati…

Autonomous Driving Formal Logic Survey Trajectory Prediction

LLM Inference Enhanced by External Knowledge: A Survey

2025-05-30 · Yu-Hsuan Lin, Qian-Hui Chen, Yi-Jie Cheng, Jia-Ren Zhang 외

Recent advancements in large language models (LLMs) have enhanced natural-language reasoning. However, their limited parametric memory and susceptibility to hallucination present persistent challenges for tasks requiring…

Hallucination Knowledge Graphs Survey