paper-with-me

홈 › Papers

Reliable and Compact Graph Fine-tuning via GraphSparse Prompting

2024-10-29 · Bo Jiang, Hao Wu, Beibei Wang, Jin Tang, Bin Luo

Recently, graph prompt learning has garnered increasing attention in adapting pre-trained GNN models for downstream graph learning tasks. However, existing works generally conduct prompting over all graph elements (e.g., nodes, edges, node attributes, etc.), which is suboptimal and obviously redundant. To address this issue, we propose exploiting sparse representation theory for graph prompting and present Graph Sparse Prompting (GSP). GSP aims to adaptively and sparsely select the optimal elements (e.g., certain node attributes) to achieve compact prompting for downstream tasks. Specifically, we propose two kinds of GSP models, termed Graph Sparse Feature Prompting (GSFP) and Graph Sparse multi-Feature Prompting (GSmFP). Both GSFP and GSmFP provide a general scheme for tuning any specific pre-trained GNNs that can achieve attribute selection and compact prompt learning simultaneously. A simple yet effective algorithm has been designed for solving GSFP and GSmFP models. Experiments on 16 widely-used benchmark datasets validate the effectiveness and advantages of the proposed GSFPs.

📄 PDF Abstract BibTeX arXiv:2410.21749

Code (0)

등록된 구현이 없습니다.

Tasks

AttributeGraph LearningPrompt Learning

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Attention 설명 없음

Similar Papers 제목 키워드 기반

GraphSparseNet: a Novel Method for Large Scale Traffic Flow Prediction

2025-02-27 · Weiyang Kong, Kaiqi Wu, Sen Zhang, Yubao Liu

Traffic flow forecasting is a critical spatio-temporal data mining task with wide-ranging applications in intelligent route planning and dynamic traffic management. Recent advancements in deep learning, particularly thro…

Scalable Heterogeneous Graph Foundation Models for Data-Driven Optimal Power Flow in Smart Grids

2026-05-22 · Massimiliano Lupo Pasini, Yijiang Li, Kibaek Kim, Teja Kuruganti arxiv

Fast and reliable optimal power flow (OPF) approximation is essential for reliable smart-grid operation, yet many learning-based surrogates either flatten the native heterogeneous structure of power networks, target a li…

Hyperparameter OptimizationGraph Neural Network

On the Robustness Tradeoff in Fine-Tuning

2025-03-19 · Kunyang Li, Jean-Charles Noirot Ferrand, Ryan Sheatsley, Blaine Hoak 외

Fine-tuning has become the standard practice for adapting pre-trained (upstream) models to downstream tasks. However, the impact on model robustness is not well understood. In this work, we characterize the robustness-ac…

Adversarial Robustness

A Unified Graph Selective Prompt Learning for Graph Neural Networks

2024-06-15 · Bo Jiang, Hao Wu, Ziyan Zhang, Beibei Wang 외

In recent years, graph prompt learning/tuning has garnered increasing attention in adapting pre-trained models for graph representation learning. As a kind of universal graph prompt learning method, Graph Prompt Feature …

Graph Representation LearningPrompt LearningRepresentation Learning

CompAct: Compressed Activations for Memory-Efficient LLM Training

2024-10-20 · Yara Shamshoum, Nitzan Hodos, Yuval Sieradzki, Assaf Schuster

We introduce CompAct, a technique that reduces peak memory utilization on GPU by 25-30% for pretraining and 50% for fine-tuning of LLMs. Peak device memory is a major limiting factor in training LLMs, with various recent…

GPU