Papers Hyperparameter Optimization
“Hyperparameter Optimization” 태그가 달린 논문 1,012편 · 필터 해제
A Data-Centric Perspective on Evaluating Machine Learning Models for Tabular Data
Tabular data is prevalent in real-world machine learning applications, and new models for supervised learning of tabular data are frequently proposed. Comparative studies assessing the performance of models typically con…
Feature EngineeringHyperparameter OptimizationModel SelectionTest-time AdaptationScalable Nested Optimization for Deep Learning
Gradient-based optimization has been critical to the success of machine learning, updating a single set of parameters to minimize a single loss. A growing number of applications rely on a generalization of this, where we…
Deep LearningHyperparameter OptimizationHyperparameter Optimization for Randomized Algorithms: A Case Study on Random Features
Randomized algorithms exploit stochasticity to reduce computational complexity. One important example is random feature regression (RFR) that accelerates Gaussian process regression (GPR). RFR approximates an unknown fun…
GPRHyperparameter OptimizationregressionFast Optimizer Benchmark
In this paper, we present the Fast Optimizer Benchmark (FOB), a tool designed for evaluating deep learning optimizers during their development. The benchmark supports tasks from multiple domains such as computer vision, …
AutoMLGraph LearningHyperparameter OptimizationImproving Hyperparameter Optimization with Checkpointed Model Weights
When training deep learning models, the performance depends largely on the selected hyperparameters. However, hyperparameter optimization (HPO) is often one of the most expensive parts of model design. Classical HPO meth…
Hyperparameter OptimizationmodelEnhancing supply chain security with automated machine learning
The increasing scale and complexity of global supply chains have led to new challenges spanning various fields, such as supply chain disruptions due to long waiting lines at the ports, material shortages, and inflation. …
feature selectionFraud DetectionHyperparameter OptimizationModel OptimizationUnder the Hood of Tabular Data Generation Models: Benchmarks with Extensive Tuning
The ability to train generative models that produce realistic, safe and useful tabular data is essential for data privacy, imputation, oversampling, explainability or simulation. However, generating tabular data is not s…
GPUHyperparameter OptimizationImputationTabular Data GenerationAnalysing Multi-Task Regression via Random Matrix Theory with Application to Time Series Forecasting
In this paper, we introduce a novel theoretical framework for multi-task regression, applying random matrix theory to provide precise performance estimations, under high-dimensional, non-Gaussian data distributions. We f…
Hyperparameter OptimizationMulti-Task LearningMultivariate Time Series Forecastingregression+2Scalable Training of Trustworthy and Energy-Efficient Predictive Graph Foundation Models for Atomistic Materials Modeling: A Case Study with HydraGNN
We present our work on developing and training scalable, trustworthy, and energy-efficient predictive graph foundation models (GFMs) using HydraGNN, a multi-headed graph convolutional neural network architecture. HydraGN…
Atomic ForcesEnsemble LearningGraph Neural NetworkHyperparameter Optimization+2Optimizing Deep Reinforcement Learning for Adaptive Robotic Arm Control
In this paper, we explore the optimization of hyperparameters for the Soft Actor-Critic (SAC) and Proximal Policy Optimization (PPO) algorithms using the Tree-structured Parzen Estimator (TPE) in the context of robotic a…
Deep Reinforcement LearningHyperparameter Optimizationreinforcement-learningReinforcement LearningAdapters Strike Back
Adapters provide an efficient and lightweight mechanism for adapting trained transformer models to a variety of different tasks. However, they have often been found to be outperformed by other adaptation mechanisms, incl…
Hyperparameter OptimizationFunBO: Discovering Acquisition Functions for Bayesian Optimization with FunSearch
The sample efficiency of Bayesian optimization algorithms depends on carefully crafted acquisition functions (AFs) guiding the sequential collection of function evaluations. The best-performing AF can vary significantly …
Bayesian Optimizationglobal-optimizationHyperparameter OptimizationTransfer LearningDistributional bias compromises leave-one-out cross-validation
Cross-validation is a common method for estimating the predictive performance of machine learning models. In a data-scarce regime, where one typically wishes to maximize the number of instances used for training the mode…
Hyperparameter OptimizationNon-uniformity is All You Need: Efficient and Timely Encrypted Traffic Classification With ECHO
With 95% of Internet traffic now encrypted, an effective approach to classifying this traffic is crucial for network security and management. This paper introduces ECHO -- a novel optimization process for ML/DL-based enc…
AllClassificationEarly ClassificationHyperparameter Optimization+1Gated recurrent neural network with TPE Bayesian optimization for enhancing stock index prediction accuracy
The recent advancement of deep learning architectures, neural networks, and the combination of abundant financial data and powerful computers are transforming finance, leading us to develop an advanced method for predict…
Bayesian Optimizationfeature selectionHyperparameter OptimizationCost-Sensitive Multi-Fidelity Bayesian Optimization with Transfer of Learning Curve Extrapolation
In this paper, we address the problem of cost-sensitive multi-fidelity Bayesian Optimization (BO) for efficient hyperparameter optimization (HPO). Specifically, we assume a scenario where users want to early-stop the BO …
Bayesian OptimizationHyperparameter OptimizationTransfer LearningReshuffling Resampling Splits Can Improve Generalization of Hyperparameter Optimization
Hyperparameter optimization is crucial for obtaining peak performance of machine learning models. The standard protocol evaluates various hyperparameter configurations using a resampling estimate of the generalization er…
Hyperparameter OptimizationA Trajectory-Based Bayesian Approach to Multi-Objective Hyperparameter Optimization with Epoch-Aware Trade-Offs
Training machine learning models inherently involves a resource-intensive and noisy iterative learning procedure that allows epoch-wise monitoring of the model performance. However, the insights gained from the iterative…
Bayesian OptimizationDecision MakingHyperparameter OptimizationDerivatives of Stochastic Gradient Descent in parametric optimization
We consider stochastic optimization problems where the objective depends on some parameter, as commonly found in hyperparameter optimization for instance. We investigate the behavior of the derivatives of the iterates of…
Hyperparameter OptimizationStochastic OptimizationExploring Public Attention in the Circular Economy through Topic Modelling with Twin Hyperparameter Optimisation
To advance the circular economy (CE), it is crucial to gain insights into the evolution of public attention, cognitive pathways of the masses concerning circular products, and to identify primary concerns. To achieve thi…
Dynamic Topic ModelingHyperparameter OptimizationTopic ModelsWord Embeddings