paper-with-me

Papers

Safe Gradient Flow for Bilevel Optimization

2025-01-27 · Sina Sharifi, Nazanin Abolfazli, Erfan Yazdandoost Hamedani, Mahyar Fazlyab

Bilevel optimization is a key framework in hierarchical decision-making, where one problem is embedded within the constraints of another. In this work, we propose a control-theoretic approach to solving bilevel optimization problems. Our method consists of two components: a gradient flow mechanism to minimize the upper-level objective and a safety filter to enforce the constraints imposed by the lower-level problem. Together, these components form a safe gradient flow that solves the bilevel problem in a single loop. To improve scalability with respect to the lower-level problem's dimensions, we introduce a relaxed formulation and design a compact variant of the safe gradient flow. This variant minimizes the upper-level objective while ensuring the lower-level decision variable remains within a user-defined suboptimality. Using Lyapunov analysis, we establish convergence guarantees for the dynamics, proving that they converge to a neighborhood of the optimal solution. Numerical experiments further validate the effectiveness of the proposed approaches. Our contributions provide both theoretical insights and practical tools for efficiently solving bilevel optimization problems.

📄 PDF Abstract BibTeX arXiv:2501.16520

Code (1)

o4lc/sgf-blo 공식 구현 pytorch

Tasks

Bilevel OptimizationDecision Making

Similar Papers 제목 키워드 기반

FlowBot: Inducing LLM Workflows with Bilevel Optimization and Textual Gradients

2026-04-29 · Hongyeon Yu, Young-Bum Kim, Yoon Kim arxiv

LLM workflows, which coordinate structured calls to individual LLMs/agents to achieve a particular goal, offer a promising path towards building powerful AI systems that can tackle diverse tasks. However, existing approa…

Bilevel Optimization

On Penalty-based Bilevel Gradient Descent Method

2023-02-10 · Han Shen, Quan Xiao, Tianyi Chen

Bilevel optimization enjoys a wide range of applications in emerging machine learning and signal processing problems such as hyper-parameter optimization, image reconstruction, meta-learning, adversarial training, and re…

Bilevel OptimizationImage ReconstructionMeta-Learningreinforcement-learning+1

Nonconvex Decentralized Stochastic Bilevel Optimization under Heavy-Tailed Noise

2025-09-19 · Xinwen Zhang, Yihan Zhang, Heng Liang, Hongchang Gao arxiv

Existing decentralized stochastic optimization methods assume the lower-level loss function is strongly convex and the stochastic gradient noise has finite variance. These strong assumptions typically are not satisfied i…

Stochastic OptimizationBilevel Optimization

Efficient Bilevel Optimization with KFAC-Based Hypergradients

2026-03-31 · Disen Liao, Felix Dangel, Yaoliang Yu arxiv

Bilevel optimization (BO) is widely applicable to many machine learning problems. Scaling BO, however, requires repeatedly computing hypergradients, which involves solving inverse Hessian-vector products (IHVPs). In prac…

Bilevel Optimization

A Single-Loop Algorithm for Decentralized Bilevel Optimization

2023-11-15 · Youran Dong, Shiqian Ma, Junfeng Yang, Chao Yin

Bilevel optimization has gained significant attention in recent years due to its broad applications in machine learning. This paper focuses on bilevel optimization in decentralized networks and proposes a novel single-lo…

Bilevel OptimizationHyperparameter Optimization