Implicit Differentiation for Hyperparameter Tuning the Weighted Graphical Lasso
We provide a framework and algorithm for tuning the hyperparameters of the Graphical Lasso via a bilevel optimization problem solved with a first-order method. In particular, we derive the Jacobian of the Graphical Lasso solution with respect to its regularization hyperparameters.
Code (0)
등록된 구현이 없습니다.
Tasks
Bilevel OptimizationSimilar Papers 제목 키워드 기반
Nonsmooth Implicit Differentiation for Machine Learning and Optimization
In view of training increasingly complex learning architectures, we establish a nonsmooth implicit function theorem with an operational calculus. Our result applies to most practical problems (i.e., definable problems) p…
BIG-bench Machine LearningNonsmooth Implicit Differentiation for Machine-Learning and Optimization
In view of training increasingly complex learning architectures, we establish a nonsmooth implicit function theorem with an operational calculus. Our result applies to most practical problems (i.e., definable problems) p…
BIG-bench Machine LearningOptimizing Millions of Hyperparameters by Implicit Differentiation
We propose an algorithm for inexpensive gradient-based hyperparameter optimization that combines the implicit function theorem (IFT) with efficient inverse Hessian approximations. We present results about the relationshi…
Data AugmentationHyperparameter OptimizationImplicit differentiation of Lasso-type models for hyperparameter optimization
Setting regularization parameters for Lasso-type estimators is notoriously difficult, though crucial in practice. The most popular hyperparameter optimization approach is grid-search using held-out validation data. Grid-…
Hyperparameter OptimizationVocal Bursts Type PredictionImplicit differentiation for fast hyperparameter selection in non-smooth convex learning
Finding the optimal hyperparameters of a model can be cast as a bilevel optimization problem, typically solved using zero-order techniques. In this work we study first-order methods when the inner optimization problem is…
Bilevel OptimizationHyperparameter Optimization