Dynamic Anisotropic Smoothing for Noisy Derivative-Free Optimization
We propose a novel algorithm that extends the methods of ball smoothing and Gaussian smoothing for noisy derivative-free optimization by accounting for the heterogeneous curvature of the objective function. The algorithm dynamically adapts the shape of the smoothing kernel to approximate the Hessian of the objective function around a local optimum. This approach significantly reduces the error in estimating the gradient from noisy evaluations through sampling. We demonstrate the efficacy of our method through numerical experiments on artificial problems. Additionally, we show improved performance when tuning NP-hard combinatorial optimization solvers compared to existing state-of-the-art heuristic derivative-free and Bayesian optimization methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian OptimizationCombinatorial OptimizationSimilar Papers 제목 키워드 기반
Directional Graph Networks
The lack of anisotropic kernels in graph neural networks (GNNs) strongly limits their expressiveness, contributing to well-known issues such as over-smoothing. To overcome this limitation, we propose the first globally c…
Data AugmentationGraph ClassificationGraph RegressionNode ClassificationCovariance-based smoothed particle hydrodynamics. A machine-learning application to simulating disc fragmentation
A PCA-based, machine learning version of the SPH method is proposed. In the present scheme, the smoothing tensor is computed to have their eigenvalues proportional to the covariance's principal components, using a modifi…
BIG-bench Machine LearningMalliavin Calculus with Weak Derivatives for Counterfactual Stochastic Optimization
We study counterfactual stochastic optimization of conditional loss functionals under misspecified and noisy gradient information. The difficulty is that when the conditioning event has vanishing or zero probability, nai…
Stochastic OptimizationSparse Inpainting with Smoothed Particle Hydrodynamics
Digital image inpainting refers to techniques used to reconstruct a damaged or incomplete image by exploiting available image information. The main goal of this work is to perform the image inpainting process from a set …
Image InpaintingSecond-order Anisotropic Gaussian Directional Derivative Filters for Blob Detection
Interest point detection methods have received increasing attention and are widely used in computer vision tasks such as image retrieval and 3D reconstruction. In this work, second-order anisotropic Gaussian directional …
3D ReconstructionImage RetrievalInterest Point DetectionRetrieval