Bilevel Graph Structure Learning, Revisited: Inner-Channel Origins of the Reported Gain
Bilevel graph structure learning is widely understood to improve graph neural networks by jointly optimizing model parameters and a learned graph structure, with the resulting performance gain attributed to the rewired adjacency. We find that this attribution may be overstated: training-dynamics effects in the inner loop, rather than the rewiring itself, capture a substantial share of the gain. To establish this, we introduce frozen-$φ$, a control that freezes the graph while retaining the inner-loop training schedule. This decomposes the bilevel gain into an inner channel of $T$-step training dynamics with implicit gradient regularization and a graph channel of the graph rewiring itself. On spatio-temporal flow forecasting the inner channel matches or exceeds the full bilevel pipeline, accounting for 78-101% of the gain; on node classification it accounts for 37-44% under a Bernoulli edge-level parameterization. We also verify that classical spectral diagnostics can dissociate from task gain. We propose frozen-$φ$ as a standardized diagnostic for bilevel graph structure learning, with graph distillation as a method-agnostic complement. A three-precondition framework further predicts the sign of the bilevel gain on all six benchmarks.
Code (0)
등록된 구현이 없습니다.
Tasks
Graph structure learningNode ClassificationSimilar Papers 제목 키워드 기반
A Primal-Dual Approach to Bilevel Optimization with Multiple Inner Minima
Bilevel optimization has found extensive applications in modern machine learning problems such as hyperparameter optimization, neural architecture search, meta-learning, etc. While bilevel problems with a unique inner mi…
Bilevel OptimizationHyperparameter OptimizationMeta-LearningNeural Architecture SearchEffective Bilevel Optimization via Minimax Reformulation
Bilevel optimization has found successful applications in various machine learning problems, including hyper-parameter optimization, data cleaning, and meta-learning. However, its huge computational cost presents a signi…
Bilevel OptimizationMeta-LearningA Fully Single Loop Algorithm for Bilevel Optimization without Hessian Inverse
In this paper, we propose a new Hessian inverse free Fully Single Loop Algorithm (FSLA) for bilevel optimization problems. Classic algorithms for bilevel optimization admit a double loop structure which is computationall…
Bilevel OptimizationFunctional Bilevel Optimization for Machine Learning
In this paper, we introduce a new functional point of view on bilevel optimization problems for machine learning, where the inner objective is minimized over a function space. These types of problems are most often solve…
Bilevel OptimizationPredictionA Two-Timescale Framework for Bilevel Optimization: Complexity Analysis and Application to Actor-Critic
This paper analyzes a two-timescale stochastic algorithm framework for bilevel optimization. Bilevel optimization is a class of problems which exhibit a two-level structure, and its goal is to minimize an outer objective…
Bilevel OptimizationHyperparameter Optimization