Bayesian Optimization for Hyperparameters Tuning in Neural Networks
This study investigates the application of Bayesian Optimization (BO) for the hyperparameter tuning of neural networks, specifically targeting the enhancement of Convolutional Neural Networks (CNN) for image classification tasks. Bayesian Optimization is a derivative-free global optimization method suitable for expensive black-box functions with continuous inputs and limited evaluation budgets. The BO algorithm leverages Gaussian Process regression and acquisition functions like Upper Confidence Bound (UCB) and Expected Improvement (EI) to identify optimal configurations effectively. Using the Ax and BOTorch frameworks, this work demonstrates the efficiency of BO in reducing the number of hyperparameter tuning trials while achieving competitive model performance. Experimental outcomes reveal that BO effectively balances exploration and exploitation, converging rapidly towards optimal settings for CNN architectures. This approach underlines the potential of BO in automating neural network tuning, contributing to improved accuracy and computational efficiency in machine learning pipelines.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian OptimizationComputational Efficiencyglobal-optimizationimage-classificationImage ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Fast Hyperparameter Tuning using Bayesian Optimization with Directional Derivatives
In this paper we develop a Bayesian optimization based hyperparameter tuning framework inspired by statistical learning theory for classifiers. We utilize two key facts from PAC learning theory; the generalization bound …
Bayesian OptimizationLearning TheoryPAC learningSmall Data Image ClassificationTowards Assessing the Impact of Bayesian Optimization's Own Hyperparameters
Bayesian Optimization (BO) is a common approach for hyperparameter optimization (HPO) in automated machine learning. Although it is well-accepted that HPO is crucial to obtain well-performing machine learning models, tun…
Bayesian OptimizationBIG-bench Machine LearningHyperparameter OptimizationNeural Architecture SearchModel Fusion through Bayesian Optimization in Language Model Fine-Tuning
Fine-tuning pre-trained models for downstream tasks is a widely adopted technique known for its adaptability and reliability across various domains. Despite its conceptual simplicity, fine-tuning entails several troubles…
Bayesian OptimizationLanguage ModelingLanguage ModellingmodelAlleviating Hyperparameter-Tuning Burden in SVM Classifiers for Pulmonary Nodules Diagnosis with Multi-Task Bayesian Optimization
In the field of non-invasive medical imaging, radiomic features are utilized to measure tumor characteristics. However, these features can be affected by the techniques used to discretize the images, ultimately impacting…
Bayesian OptimizationTowards Automatic Bayesian Optimization: A first step involving acquisition functions
Bayesian Optimization is the state of the art technique for the optimization of black boxes, i.e., functions where we do not have access to their analytical expression nor its gradients, they are expensive to evaluate an…
Bayesian OptimizationBIG-bench Machine Learning