A survey on multi-objective hyperparameter optimization algorithms for Machine Learning

2021-11-23 · Alejandro Morales-Hernández, Inneke Van Nieuwenhuyse, Sebastian Rojas Gonzalez

Hyperparameter optimization (HPO) is a necessary step to ensure the best possible performance of Machine Learning (ML) algorithms. Several methods have been developed to perform HPO; most of these are focused on optimizing one performance measure (usually an error-based measure), and the literature on such single-objective HPO problems is vast. Recently, though, algorithms have appeared that focus on optimizing multiple conflicting objectives simultaneously. This article presents a systematic survey of the literature published between 2014 and 2020 on multi-objective HPO algorithms, distinguishing between metaheuristic-based algorithms, metamodel-based algorithms, and approaches using a mixture of both. We also discuss the quality metrics used to compare multi-objective HPO procedures and present future research directions.

📄 PDF Abstract BibTeX arXiv:2111.13755

Code (0)

등록된 구현이 없습니다.

Tasks

BIG-bench Machine Learning Hyperparameter Optimization

Methods 이 논문이 사용한 방법론

HPO In machine learning, a hyperparameter is a parameter whose value is used to control learning process, and HPO is the problem of choosing a set of optimal hyperparameters for a…

Multi-Objective Hyperparameter Optimization in Machine Learning -- An Overview

2022-06-15 · Florian Karl, Tobias Pielok, Julia Moosbauer, Florian Pfisterer 외

Hyperparameter optimization constitutes a large part of typical modern machine learning workflows. This arises from the fact that machine learning methods and corresponding preprocessing steps often only yield optimal pe…

Bayesian Optimization BIG-bench Machine Learning Evolutionary Algorithms Fairness+1

Hyperparameter Optimization in Machine Learning

2024-10-30 · Luca Franceschi, Michele Donini, Valerio Perrone, Aaron Klein 외

Hyperparameters are configuration variables controlling the behavior of machine learning algorithms. They are ubiquitous in machine learning and artificial intelligence and the choice of their values determines the effec…

Hyperparameter Optimization Meta-Learning Neural Architecture Search

Fair and Green Hyperparameter Optimization via Multi-objective and Multiple Information Source Bayesian Optimization

2022-05-18 · Antonio Candelieri, Andrea Ponti, Francesco Archetti

There is a consensus that focusing only on accuracy in searching for optimal machine learning models amplifies biases contained in the data, leading to unfair predictions and decision supports. Recently, multi-objective …

Bayesian Optimization BIG-bench Machine Learning Fairness Hyperparameter Optimization

Experimental Investigation and Evaluation of Model-based Hyperparameter Optimization

2021-07-19 · Eva Bartz, Martin Zaefferer, Olaf Mersmann, Thomas Bartz-Beielstein

Machine learning algorithms such as random forests or xgboost are gaining more importance and are increasingly incorporated into production processes in order to enable comprehensive digitization and, if possible, automa…

BIG-bench Machine Learning Hyperparameter Optimization

Multi-objective Hyperparameter Optimization in the Age of Deep Learning

2025-11-11 · Soham Basu, Frank Hutter, Danny Stoll arxiv

While Deep Learning (DL) experts often have prior knowledge about which hyperparameter settings yield strong performance, only few Hyperparameter Optimization (HPO) algorithms can leverage such prior knowledge and none i…

Hyperparameter Optimization