Adaptive Linear Embedding for Nonstationary High-Dimensional Optimization
Bayesian Optimization (BO) in high-dimensional spaces remains fundamentally limited by the curse of dimensionality and the rigidity of global low-dimensional assumptions. While Random EMbedding Bayesian Optimization (REMBO) mitigates this via linear projections into low-dimensional subspaces, it typically assumes a single global embedding and a stationary objective. In this work, we introduce Self-Adaptive embedding REMBO (SA-REMBO), a novel framework that generalizes REMBO to support multiple random Gaussian embeddings, each capturing a different local subspace structure of the high-dimensional objective. An index variable governs the embedding choice and is jointly modeled with the latent optimization variable via a product kernel in a Gaussian Process surrogate. This enables the optimizer to adaptively select embeddings conditioned on location, effectively capturing locally varying effective dimensionality, nonstationarity, and heteroscedasticity in the objective landscape. We theoretically analyze the expressiveness and stability of the index-conditioned product kernel and empirically demonstrate the advantage of our method across synthetic and real-world high-dimensional benchmarks, where traditional REMBO and other low-rank BO methods fail. Our results establish SA-REMBO as a powerful and flexible extension for scalable BO in complex, structured design spaces.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian OptimizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
AdaKoop: Efficient Modeling of Nonlinear Dynamics from Nonstationary Data Streams with Koopman Operator Regression
Real-time data analysis requires the ability to accurately and adaptively address nonlinear dynamics in a nonstationary data stream while preserving computational efficiency. However, nonlinear dynamics are so complex th…
Computational EfficiencyProvably Efficient Algorithm for Nonstationary Low-Rank MDPs
Reinforcement learning (RL) under changing environment models many real-world applications via nonstationary Markov Decision Processes (MDPs), and hence gains considerable interest. However, theoretical studies on nonsta…
Reinforcement Learning (RL)A Kernel Embedding-based Approach for Nonstationary Causal Model Inference
Although nonstationary data are more common in the real world, most existing causal discovery methods do not take nonstationarity into consideration. In this letter, we propose a kernel embedding-based approach, ENCI, fo…
Causal DiscoveryTwo-Dimensional Nonstationary Filtering by Operator Scaling
f–k filtering and Radon transform (RT) are classical methods for processing 2-D seismic signals. They assume that target signal events exhibit specific trajectories in the time–offset domain (linear, parabolic, and hyper…
GeophysicsSpatial Mapping with Gaussian Processes and Nonstationary Fourier Features
The use of covariance kernels is ubiquitous in the field of spatial statistics. Kernels allow data to be mapped into high-dimensional feature spaces and can thus extend simple linear additive methods to nonlinear methods…
Gaussian ProcessesTime SeriesTime Series Analysis