Papers Hyperparameter Optimization
“Hyperparameter Optimization” 태그가 달린 논문 1,012편 · 필터 해제
On the Iteration Complexity of Hypergradient Computation
We study a general class of bilevel problems, consisting in the minimization of an upper-level objective which depends on the solution to a parametric fixed-point equation. Important instances arising in machine learning…
Computational EfficiencyHyperparameter OptimizationMeta-LearningSimple and Scalable Parallelized Bayesian Optimization
In recent years, leveraging parallel and distributed computational resources has become essential to solve problems of high computational cost. Bayesian optimization (BO) has shown attractive results in those expensive-t…
Bayesian OptimizationBIG-bench Machine LearningHyperparameter OptimizationEfficient Hyperparameter Optimization in Deep Learning Using a Variable Length Genetic Algorithm
Convolutional Neural Networks (CNN) have gained great success in many artificial intelligence tasks. However, finding a good set of hyperparameters for a CNN remains a challenging task. It usually takes an expert with de…
CPUHyperparameter OptimizationEfficient Hyperparameter Optimization under Multi-Source Covariate Shift
A typical assumption in supervised machine learning is that the train (source) and test (target) datasets follow completely the same distribution. This assumption is, however, often violated in uncertain real-world appli…
Bayesian OptimizationHyperparameter OptimizationRanking and benchmarking framework for sampling algorithms on synthetic data streams
In the fields of big data, AI, and streaming processing, we work with large amounts of data from multiple sources. Due to memory and network limitations, we process data streams on distributed systems to alleviate comput…
BenchmarkingHyperparameter OptimizationThe Statistical Cost of Robust Kernel Hyperparameter Tuning
This paper studies the statistical complexity of kernel hyperparameter tuning in the setting of active regression under adversarial noise. We consider the problem of finding the best interpolant from a class of kernels w…
Hyperparameter OptimizationregressionAutoHAS: Efficient Hyperparameter and Architecture Search
Efficient hyperparameter or architecture search methods have shown remarkable results, but each of them is only applicable to searching for either hyperparameters (HPs) or architectures. In this work, we propose a unifie…
AutoMLHyperparameter OptimizationNeural Architecture SearchReinforcement Learning (RL)UFO-BLO: Unbiased First-Order Bilevel Optimization
Bilevel optimization (BLO) is a popular approach with many applications including hyperparameter optimization, neural architecture search, adversarial robustness and model-agnostic meta-learning. However, the approach su…
Adversarial RobustnessBilevel OptimizationHyperparameter OptimizationMeta-Learning+2Geometric Graph Representations and Geometric Graph Convolutions for Deep Learning on Three-Dimensional (3D) Graphs
The geometry of three-dimensional (3D) graphs, consisting of nodes and edges, plays a crucial role in many important applications. An excellent example is molecular graphs, whose geometry influences important properties …
Hyperparameter OptimizationHyperparameter optimization with REINFORCE and Transformers
Reinforcement Learning has yielded promising results for Neural Architecture Search (NAS). In this paper, we demonstrate how its performance can be improved by using a simplified Transformer block to model the policy net…
BenchmarkingHyperparameter OptimizationNeural Architecture SearchQ-LearningSemi-supervised Embedding Learning for High-dimensional Bayesian Optimization
Bayesian optimization is a broadly applied methodology to optimize the expensive black-box function. Despite its success, it still faces the challenge from the high-dimensional search space. To alleviate this problem, we…
Bayesian OptimizationDimensionality ReductionHyperparameter OptimizationVocal Bursts Intensity PredictionMANGO: A Python Library for Parallel Hyperparameter Tuning
Tuning hyperparameters for machine learning algorithms is a tedious task, one that is typically done manually. To enable automated hyperparameter tuning, recent works have started to use techniques based on Bayesian opti…
Bayesian OptimizationBIG-bench Machine LearningDistributed ComputingDistributed Optimization+2HyperSTAR: Task-Aware Hyperparameters for Deep Networks
While deep neural networks excel in solving visual recognition tasks, they require significant effort to find hyperparameters that make them work optimally. Hyperparameter Optimization (HPO) approaches have automated the…
Hyperparameter Optimizationimage-classificationImage ClassificationSherpa: Robust Hyperparameter Optimization for Machine Learning
Sherpa is a hyperparameter optimization library for machine learning models. It is specifically designed for problems with computationally expensive, iterative function evaluations, such as the hyperparameter tuning of d…
BIG-bench Machine LearningHyperparameter OptimizationExploring the Loss Landscape in Neural Architecture Search
Neural architecture search (NAS) has seen a steep rise in interest over the last few years. Many algorithms for NAS consist of searching through a space of architectures by iteratively choosing an architecture, evaluatin…
Combinatorial OptimizationDenoisingHyperparameter OptimizationNeural Architecture Search+1Frugal Optimization for Cost-related Hyperparameters
The increasing demand for democratizing machine learning algorithms calls for hyperparameter optimization (HPO) solutions at low cost. Many machine learning algorithms have hyperparameters which can cause a large variati…
AutoMLBIG-bench Machine LearningHyperparameter OptimizationYou CAN Teach an Old Dog New Tricks! On Training Knowledge Graph Embeddings
Knowledge graph embedding (KGE) models learn algebraic representations of the entities and relations in a knowledge graph. A vast number of KGE techniques for multi-relational link prediction have been proposed in the re…
Graph EmbeddingHyperparameter OptimizationKnowledge Graph EmbeddingKnowledge Graph Embeddings+1Hyperparameter Optimization in Binary Communication Networks for Neuromorphic Deployment
Training neural networks for neuromorphic deployment is non-trivial. There have been a variety of approaches proposed to adapt back-propagation or back-propagation-like algorithms appropriate for training. Considering th…
Hyperparameter OptimizationMeta-Learning in Neural Networks: A Survey
The field of meta-learning, or learning-to-learn, has seen a dramatic rise in interest in recent years. Contrary to conventional approaches to AI where tasks are solved from scratch using a fixed learning algorithm, meta…
Few-Shot LearningHyperparameter OptimizationMeta-LearningMulti-Task Learning+3Direct loss minimization algorithms for sparse Gaussian processes
The paper provides a thorough investigation of Direct loss minimization (DLM), which optimizes the posterior to minimize predictive loss, in sparse Gaussian processes. For the conjugate case, we consider DLM for log-loss…
Computational EfficiencyGaussian ProcessesHyperparameter OptimizationStochastic Optimization+1