Hyperparameter Tuning Cookbook: A guide for scikit-learn, PyTorch, river, and spotPython
This document provides a comprehensive guide to hyperparameter tuning using spotPython for scikit-learn, PyTorch, and river. The first part introduces spotPython's surrogate model-based optimization process, while the second part focuses on hyperparameter tuning. Several case studies are presented, including hyperparameter tuning for sklearn models such as Support Vector Classification, Random Forests, Gradient Boosting (XGB), and K-nearest neighbors (KNN), as well as a Hoeffding Adaptive Tree Regressor from river. The integration of spotPython into the PyTorch and PyTorch Lightning training workflow is also discussed. With a hands-on approach and step-by-step explanations, this cookbook serves as a practical starting point for anyone interested in hyperparameter tuning with Python. Highlights include the interplay between Tensorboard, PyTorch Lightning, spotPython, and river. This publication is under development, with updates available on the corresponding webpage.
Code (1)
Similar Papers 제목 키워드 기반
scikit-fda: A Python Package for Functional Data Analysis
The library scikit-fda is a Python package for Functional Data Analysis (FDA). It provides a comprehensive set of tools for representation, preprocessing, and exploratory analysis of functional data. The library is built…
Model SelectionMANGO: A Python Library for Parallel Hyperparameter Tuning
Tuning hyperparameters for machine learning algorithms is a tedious task, one that is typically done manually. To enable automated hyperparameter tuning, recent works have started to use techniques based on Bayesian opti…
Bayesian OptimizationBIG-bench Machine LearningDistributed ComputingDistributed Optimization+2Cookbook: A framework for improving LLM generative abilities via programmatic data generating templates
Fine-tuning large language models (LLMs) on instruction datasets is a common way to improve their generative capabilities. However, instruction datasets can be expensive and time-consuming to manually curate, and while L…
Default Machine Learning Hyperparameters Do Not Provide Informative Initialization for Bayesian Optimization
Bayesian Optimization (BO) is a standard tool for hyperparameter tuning thanks to its sample efficiency on expensive black-box functions. While most BO pipelines begin with uniform random initialization, default hyperpar…
Interim Report on Human-Guided Adaptive Hyperparameter Optimization with Multi-Fidelity Sprints
This case study applies a phased hyperparameter optimization process to compare multitask natural language model variants that utilize multiphase learning rate scheduling and optimizer parameter grouping. We employ short…
Bayesian OptimizationHyperparameter OptimizationJoint Entity and Relation ExtractionLanguage Modeling+3