Generalization and Representational Limits of Graph Neural Networks
We address two fundamental questions about graph neural networks (GNNs). First, we prove that several important graph properties cannot be computed by GNNs that rely entirely on local information. Such GNNs include the standard message passing models, and more powerful spatial variants that exploit local graph structure (e.g., via relative orientation of messages, or local port ordering) to distinguish neighbors of each node. Our treatment includes a novel graph-theoretic formalism. Second, we provide the first data dependent generalization bounds for message passing GNNs. This analysis explicitly accounts for the local permutation invariance of GNNs. Our bounds are much tighter than existing VC-dimension based guarantees for GNNs, and are comparable to Rademacher bounds for recurrent neural networks.
Code (0)
등록된 구현이 없습니다.
Tasks
Generalization BoundsSimilar Papers 제목 키워드 기반
Detecting Explanatory Insufficiency in Learned Representations: A Framework for Representational Vigilance
Learned representations are central to modern machine learning, but predictive performance, robustness, uncertainty estimation, and generalization do not by themselves establish representational adequacy. A model may rem…
Going beyond persistent homology using persistent homology
Representational limits of message-passing graph neural networks (MP-GNNs), e.g., in terms of the Weisfeiler-Leman (WL) test for isomorphism, are well understood. Augmenting these graph models with topological features v…
Graph ClassificationBootstrap Theory of Representational Emergence (TBER): Explanatory Insufficiency, Transition Regimes, and the Emergence of New Representational Levels
Representation learning is central to modern machine learning, yet most research focuses on optimizing representations after a framework has been selected. The Bootstrap Theory of Representational Emergence (TBER) addres…
Representation LearningAnomaly DetectionGraph Representational Learning: When Does More Expressivity Hurt Generalization?
Graph Neural Networks (GNNs) are powerful tools for learning on structured data, yet the relationship between their expressivity and predictive performance remains unclear. We introduce a family of premetrics that captur…
Generalization BoundsOn the Intrinsic Limits to Representationally-Adaptive Machine-Learning
Online learning is a familiar problem setting within Machine-Learning in which data is presented serially in time to a learning agent, requiring it to progressively adapt within the constraints of the learning algorithm.…
BIG-bench Machine LearningTransfer Learning