Can Language Models Capture Graph Semantics? From Graphs to Language Model and Vice-Versa
Knowledge Graphs are a great resource to capture semantic knowledge in terms of entities and relationships between the entities. However, current deep learning models takes as input distributed representations or vectors. Thus, the graph is compressed in a vectorized representation. We conduct a study to examine if the deep learning model can compress a graph and then output the same graph with most of the semantics intact. Our experiments show that Transformer models are not able to express the full semantics of the input knowledge graph. We find that this is due to the disparity between the directed, relationship and type based information contained in a Knowledge Graph and the fully connected token-token undirected graphical interpretation of the Transformer Attention matrix.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep LearningKnowledge GraphsLanguage ModelingLanguage ModellingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
The KG-ER Conceptual Schema Language
We propose KG-ER, a conceptual schema language for knowledge graphs that describes the structure of knowledge graphs independently of their representation (relational databases, property graphs, RDF) while helping to cap…
Knowledge GraphsJoint Language Semantic and Structure Embedding for Knowledge Graph Completion
The task of completing knowledge triplets has broad downstream applications. Both structural and semantic information plays an important role in knowledge graph completion. Unlike previous approaches that rely on either …
Knowledge Graph CompletionKnowledge GraphsLink PredictionEnhancing Collaborative Semantics of Language Model-Driven Recommendations via Graph-Aware Learning
Large Language Models (LLMs) are increasingly prominent in the recommendation systems domain. Existing studies usually utilize in-context learning or supervised fine-tuning on task-specific data to align LLMs into recomm…
In-Context LearningLanguage ModelingLanguage ModellingRecommendation SystemsGRENADE: Graph-Centric Language Model for Self-Supervised Representation Learning on Text-Attributed Graphs
Self-supervised representation learning on text-attributed graphs, which aims to create expressive and generalizable representations for various downstream tasks, has received increasing research attention lately. Howeve…
Contrastive LearningGraph Neural NetworkLanguage ModelingLanguage Modelling+2Global-Recent Semantic Reasoning on Dynamic Text-Attributed Graphs with Large Language Models
Dynamic Text-Attribute Graphs (DyTAGs), characterized by time-evolving graph interactions and associated text attributes, are prevalent in real-world applications. Existing methods, such as Graph Neural Networks (GNNs) a…