GaLoRA: Parameter-Efficient Graph-Aware LLMs for Node Classification
The rapid rise of large language models (LLMs) and their ability to capture semantic relationships has led to their adoption in a wide range of applications. Text-attributed graphs (TAGs) are a notable example where LLMs can be combined with Graph Neural Networks to improve the performance of node classification. In TAGs, each node is associated with textual content and such graphs are commonly seen in various domains such as social networks, citation graphs, recommendation systems, etc. Effectively learning from TAGs would enable better representations of both structural and textual representations of the graph and improve decision-making in relevant domains. We present GaLoRA, a parameter-efficient framework that integrates structural information into LLMs. GaLoRA demonstrates competitive performance on node classification tasks with TAGs, performing on par with state-of-the-art models with just 0.24% of the parameter count required by full LLM fine-tuning. We experiment with three real-world datasets to showcase GaLoRA's effectiveness in combining structural and semantical information on TAGs.
Code (0)
등록된 구현이 없습니다.
Tasks
Recommendation SystemsNode ClassificationSimilar Papers 제목 키워드 기반
Parameter-Efficient Tuning Large Language Models for Graph Representation Learning
Text-rich graphs, which exhibit rich textual information on nodes and edges, are prevalent across a wide range of real-world business applications. Large Language Models (LLMs) have demonstrated remarkable abilities in u…
Graph Neural NetworkGraph Representation LearningLink Predictionparameter-efficient fine-tuning+1Glance for Context: Learning When to Leverage LLMs for Node-Aware GNN-LLM Fusion
Learning on text-attributed graphs has motivated the use of Large Language Models (LLMs) for graph learning. However, most fusion strategies are applied uniformly across all nodes and attain only small overall performanc…
Graph LearningTeaching LLMs to See Graphs: Unifying Text and Structural Reasoning
Using Large Language Models (LLMs) to process graph-structured data is an active research area, yet current state-of-the-art approaches typically rely on multi-step pipelines with Graph Neural Network (GNN) encoders that…
Graph Neural NetworkAdversarial Attacks and Defenses on Graph-aware Large Language Models (LLMs)
Large Language Models (LLMs) are increasingly integrated with graph-structured data for tasks like node classification, a domain traditionally dominated by Graph Neural Networks (GNNs). While this integration leverages r…
Node ClassificationAdversarial AttackCan LLMs Convert Graphs to Text-Attributed Graphs?
Graphs are ubiquitous data structures found in numerous real-world applications, such as drug discovery, recommender systems, and social network analysis. Graph neural networks (GNNs) have become a popular tool to learn …
Drug DiscoveryRecommendation Systems