Papers Hyperparameter Optimization
“Hyperparameter Optimization” 태그가 달린 논문 1,012편 · 필터 해제
Batch Multi-Fidelity Bayesian Optimization with Deep Auto-Regressive Networks
Bayesian optimization (BO) is a powerful approach for optimizing black-box, expensive-to-evaluate functions. To enable a flexible trade-off between the cost and accuracy, many applications allow the function to be evalua…
Bayesian OptimizationDiversityHyperparameter OptimizationAn Empirical Study on Hyperparameter Optimization for Fine-Tuning Pre-trained Language Models
The performance of fine-tuning pre-trained language models largely depends on the hyperparameter configuration. In this paper, we investigate the performance of modern hyperparameter optimization methods (HPO) on fine-tu…
Hyperparameter OptimizationHPO-B: A Large-Scale Reproducible Benchmark for Black-Box HPO based on OpenML
Hyperparameter optimization (HPO) is a core problem for the machine learning community and remains largely unsolved due to the significant computational resources required to evaluate hyperparameter configurations. As a …
Hyperparameter OptimizationTransfer LearningMeta-Learning for Symbolic Hyperparameter Defaults
Hyperparameter optimization in machine learning (ML) deals with the problem of empirically learning an optimal algorithm configuration from data, usually formulated as a black-box optimization problem. In this work, we p…
Hyperparameter OptimizationMeta-LearningA Nonmyopic Approach to Cost-Constrained Bayesian Optimization
Bayesian optimization (BO) is a popular method for optimizing expensive-to-evaluate black-box functions. BO budgets are typically given in iterations, which implicitly assumes each evaluation has the same cost. In fact, …
Bayesian OptimizationHyperparameter OptimizationFederated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing
Tuning hyperparameters is a crucial but arduous part of the machine learning pipeline. Hyperparameter optimization is even more challenging in federated learning, where models are learned over a distributed network of he…
Federated LearningHyperparameter OptimizationNeural Architecture SearchStability and Generalization of Bilevel Programming in Hyperparameter Optimization
The (gradient-based) bilevel programming framework is widely used in hyperparameter optimization and has achieved excellent performance empirically. Previous theoretical work mainly focuses on its optimization properties…
Hyperparameter OptimizationProvably Faster Algorithms for Bilevel Optimization
Bilevel optimization has been widely applied in many important machine learning applications such as hyperparameter optimization and meta-learning. Recently, several momentum-based algorithms have been proposed to solve …
Bilevel OptimizationHyperparameter OptimizationMeta-Learningk-Mixup Regularization for Deep Learning via Optimal Transport
Mixup is a popular regularization technique for training deep neural networks that improves generalization and increases robustness to certain distribution shifts. It perturbs input training data in the direction of othe…
Adversarial RobustnessDeep LearningHyperparameter OptimizationSHINE: SHaring the INverse Estimate from the forward pass for bi-level optimization and implicit models
In recent years, implicit deep learning has emerged as a method to increase the effective depth of deep neural networks. While their training is memory-efficient, they are still significantly slower to train than their e…
Hyperparameter OptimizationBERT Goes Brrr: A Venture Towards the Lesser Error in Classifying Medical Self-Reporters on Twitter
This paper describes our team’s submission for the Social Media Mining for Health (SMM4H) 2021 shared task. We participated in three subtasks: Classifying adverse drug effect, COVID-19 self-report, and COVID-19 symptoms.…
Hyperparameter OptimizationThe Imaginative Generative Adversarial Network: Automatic Data Augmentation for Dynamic Skeleton-Based Hand Gesture and Human Action Recognition
Deep learning approaches deliver state-of-the-art performance in recognition of spatiotemporal human motion data. However, one of the main challenges in these recognition tasks is limited available training data. Insuffi…
Action RecognitionData AugmentationGenerative Adversarial NetworkGPU+2Techniques Toward Optimizing Viewability in RTB Ad Campaigns Using Reinforcement Learning
Reinforcement learning (RL) is an effective technique for training decision-making agents through interactions with their environment. The advent of deep learning has been associated with highly notable successes with se…
Bayesian OptimizationDecision MakingHyperparameter Optimizationreinforcement-learning+2DEHB: Evolutionary Hyperband for Scalable, Robust and Efficient Hyperparameter Optimization
Modern machine learning algorithms crucially rely on several design decisions to achieve strong performance, making the problem of Hyperparameter Optimization (HPO) more important than ever. Here, we combine the advantag…
Hyperparameter OptimizationNeural Architecture SearchTowards Explaining Hyperparameter Optimization via Partial Dependence Plots
Automated hyperparameter optimization (HPO) can support practitioners to obtain peak performance in machine learning models. However, there is often a lack of valuable insights into the effects of different hyperparamete…
Bayesian OptimizationBIG-bench Machine LearningHyperparameter OptimizationInterpretable Machine LearningLeveraging Theoretical Tradeoffs in Hyperparameter Selection for Improved Empirical Performance
The tradeoffs in the excess risk incurred from data-driven learning of a single model has been studied by decomposing the excess risk into approximation, estimation and optimization errors. In this paper, we focus on the…
Hyperparameter OptimizationImplicit differentiation for fast hyperparameter selection in non-smooth convex learning
Finding the optimal hyperparameters of a model can be cast as a bilevel optimization problem, typically solved using zero-order techniques. In this work we study first-order methods when the inner optimization problem is…
Bilevel OptimizationHyperparameter OptimizationBag of Baselines for Multi-objective Joint Neural Architecture Search and Hyperparameter Optimization
Neural architecture search (NAS) and hyperparameter optimization (HPO) make deep learning accessible to non-experts by automatically finding the architecture of the deep neural network to use and tuning the hyperparamete…
AutoMLHyperparameter OptimizationNeural Architecture SearchOne Billion Audio Sounds from GPU-enabled Modular Synthesis
We release synth1B1, a multi-modal audio corpus consisting of 1 billion 4-second synthesized sounds, paired with the synthesis parameters used to generate them. The dataset is 100x larger than any audio dataset in the li…
GPUHyperparameter OptimizationEfficient Hyperparameter Optimization for Physics-based Character Animation
Physics-based character animation has seen significant advances in recent years with the adoption of Deep Reinforcement Learning (DRL). However, DRL-based learning methods are usually computationally expensive and their …
Bayesian OptimizationDeep Reinforcement LearningHyperparameter Optimization