paper-with-me

홈 › Papers

Contextualized Messages Boost Graph Representations

2024-03-19 · Brian Godwin Lim, Galvin Brice Lim, Renzo Roel Tan, Kazushi Ikeda

Graph neural networks (GNNs) have gained significant attention in recent years for their ability to process data that may be represented as graphs. This has prompted several studies to explore their representational capability based on the graph isomorphism task. These works inherently assume a countable node feature representation, potentially limiting their applicability. Interestingly, only a few study GNNs with uncountable node feature representation. In the paper, a novel perspective on the representational capability of GNNs is investigated across all levels$\unicode{x2014}$node-level, neighborhood-level, and graph-level$\unicode{x2014}$when the space of node feature representation is uncountable. More specifically, the strict injective and metric requirements are softly relaxed by employing a pseudometric distance on the space of input to create a soft-injective function such that distinct inputs may produce similar outputs if and only if the pseudometric deems the inputs to be sufficiently similar on some representation. As a consequence, a simple and computationally efficient soft-isomorphic relational graph convolution network (SIR-GCN) that emphasizes the contextualized transformation of neighborhood feature representations via anisotropic and dynamic message functions is proposed. A mathematical discussion on the relationship between SIR-GCN and widely used GNNs is then laid out to put the contribution into context, establishing SIR-GCN as a generalization of classical GNN methodologies. Experiments on synthetic and benchmark datasets then demonstrate the relative superiority of SIR-GCN, outperforming comparable models in node and graph property prediction tasks.

📄 PDF Abstract BibTeX arXiv:2403.12529

Code (1)

briangodwinlim/sir-gcn 공식 구현 pytorch

Tasks

Graph Property PredictionProperty Prediction

Methods 이 논문이 사용한 방법론

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Similar Papers 제목 키워드 기반

Hierarchical Character-Word Models for Language Identification

2016-08-10 · WS 2016 11 · Aaron Jaech, George Mulcaire, Shobhit Hathi, Mari Ostendorf 외

Social media messages' brevity and unconventional spelling pose a challenge to language identification. We introduce a hierarchical model that learns character and contextualized word-level representations for language i…

Language Identification

CoreGen: Contextualized Code Representation Learning for Commit Message Generation

2020-07-14 · Lun Yiu Nie, Cuiyun Gao, Zhicong Zhong, Wai Lam 외

Automatic generation of high-quality commit messages for code commits can substantially facilitate software developers' works and coordination. However, the semantic gap between source code and natural language poses a m…

Representation LearningText Generation

Reconstruction Probing

2022-12-21 · Najoung Kim, Jatin Khilnani, Alex Warstadt, Abed Qaddoumi

We propose reconstruction probing, a new analysis method for contextualized representations based on reconstruction probabilities in masked language models (MLMs). This method relies on comparing the reconstruction proba…

Improving Machine Reading Comprehension with Contextualized Commonsense Knowledge

2020-09-12 · ACL 2022 5 · Kai Sun, Dian Yu, Jianshu Chen, Dong Yu 외

In this paper, we aim to extract commonsense knowledge to improve machine reading comprehension. We propose to represent relations implicitly by situating structured knowledge in a context instead of relying on a pre-def…

Machine Reading ComprehensionReading Comprehension

Enhancing Graph Representations with Neighborhood-Contextualized Message-Passing

2025-11-14 · Brian Godwin Lim, Galvin Brice Lim, Renzo Roel Tan, Irwin King 외 arxiv

Graph neural networks (GNNs) have become an indispensable tool for analyzing relational data. Classical GNNs are broadly classified into three variants: convolutional, attentional, and message-passing. While the standard…