Far-HO: A Bilevel Programming Package for Hyperparameter Optimization and Meta-Learning
In (Franceschi et al., 2018) we proposed a unified mathematical framework, grounded on bilevel programming, that encompasses gradient-based hyperparameter optimization and meta-learning. We formulated an approximate version of the problem where the inner objective is solved iteratively, and gave sufficient conditions ensuring convergence to the exact problem. In this work we show how to optimize learning rates, automatically weight the loss of single examples and learn hyper-representations with Far-HO, a software package based on the popular deep learning framework TensorFlow that allows to seamlessly tackle both HO and ML problems.
Code (2)
Tasks
Hyperparameter OptimizationMeta-LearningSimilar Papers 제목 키워드 기반
Bilevel Programming for Hyperparameter Optimization and Meta-Learning
We introduce a framework based on bilevel programming that unifies gradient-based hyperparameter optimization and meta-learning. We show that an approximate version of the bilevel problem can be solved by taking into exp…
Few-Shot LearningHyperparameter OptimizationMeta-LearningOptimizing Hyperparameters in CNNs using Bilevel Programming in Time Series Data
Hyperparameter optimization has remained a central topic within the machine learning community due to its ability to produce state-of-the-art results. With the recent interest growing in the usage of CNNs for time series…
BIG-bench Machine LearningHyperparameter OptimizationPositionPrediction+3Bilevel Optimization for Neural Architecture Search
Bilevel optimization has become an influential and widely adopted framework for addressing hierarchical optimization problems in machine learning, providing an effective approach to modeling the interaction between two l…
Hyperparameter OptimizationNeural Architecture SearchBilevel OptimizationStability and Generalization of Bilevel Programming in Hyperparameter Optimization
The (gradient-based) bilevel programming framework is widely used in hyperparameter optimization and has achieved excellent performance empirically. Previous theoretical work mainly focuses on its optimization properties…
Hyperparameter OptimizationOnline Nonconvex Bilevel Optimization with Bregman Divergences
Bilevel optimization methods are increasingly relevant within machine learning, especially for tasks such as hyperparameter optimization and meta-learning. Compared to the offline setting, online bilevel optimization (OB…
Bilevel OptimizationHyperparameter OptimizationMeta-Learning