Covariance Matrix Adaptation Evolution Strategy for Low Effective Dimensionality
Despite the state-of-the-art performance of the covariance matrix adaptation evolution strategy (CMA-ES), high-dimensional black-box optimization problems are challenging tasks. Such problems often involve a property called low effective dimensionality (LED), in which the objective function is formulated with redundant dimensions relative to the intrinsic objective function and a rotation transformation of the search space. The CMA-ES suffers from LED for two reasons: the default hyperparameter setting is determined by the total number of dimensions, and the norm calculations in step-size adaptations are performed including elements on the redundant dimensions. In this paper, we incorporate countermeasures for LED into the CMA-ES and propose CMA-ES-LED. We tackle with the rotation transformation using the eigenvectors of the covariance matrix. We estimate the effectiveness of each dimension in the rotated search space using the element-wise signal-to-noise ratios of the mean vector update and the rank-$\mu$ update, both of which updates can be explained as the natural gradient ascent. Then, we adapt the hyperparameter using the estimated number of effective dimensions. In addition, we refine the cumulative step-size adaptation and the two-point step-size adaptation to measure the norms only on the effective dimensions. The experimental results show the CMA-ES-LED outperforms the CMA-ES on benchmark functions with LED.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Covariance Matrix Adaptation Evolution Strategy without a matrix
Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is a highly effective optimization technique. A primary challenge when applying CMA-ES in high dimensionality is sampling from a multivariate normal distribution w…
The Hessian Estimation Evolution Strategy
We present a novel black box optimization algorithm called Hessian Estimation Evolution Strategy. The algorithm updates the covariance matrix of its sampling distribution by directly estimating the curvature of the objec…
A Simple Yet Efficient Rank One Update for Covariance Matrix Adaptation
In this paper, we propose an efficient approximated rank one update for covariance matrix adaptation evolution strategy (CMA-ES). It makes use of two evolution paths as simple as that of CMA-ES, while avoiding the comput…
Limited-Memory Matrix Adaptation for Large Scale Black-box Optimization
The Covariance Matrix Adaptation Evolution Strategy (CMA-ES) is a popular method to deal with nonconvex and/or stochastic optimization problems when the gradient information is not available. Being based on the CMA-ES, t…
Stochastic OptimizationVariable Metric Evolution Strategies for High-dimensional Multi-Objective Optimization
We design a class of variable metric evolution strategies well suited for high-dimensional problems. We target problems with many variables, not (necessarily) with many objectives. The construction combines two independe…