Policy Manifold Search: Exploring the Manifold Hypothesis for Diversity-based Neuroevolution
Neuroevolution is an alternative to gradient-based optimisation that has the potential to avoid local minima and allows parallelisation. The main limiting factor is that usually it does not scale well with parameter space dimensionality. Inspired by recent work examining neural network intrinsic dimension and loss landscapes, we hypothesise that there exists a low-dimensional manifold, embedded in the policy network parameter space, around which a high-density of diverse and useful policies are located. This paper proposes a novel method for diversity-based policy search via Neuroevolution, that leverages learned representations of the policy network parameters, by performing policy search in this learned representation space. Our method relies on the Quality-Diversity (QD) framework which provides a principled approach to policy search, and maintains a collection of diverse policies, used as a dataset for learning policy representations. Further, we use the Jacobian of the inverse-mapping function to guide the search in the representation space. This ensures that the generated samples remain in the high-density regions, after mapping back to the original space. Finally, we evaluate our contributions on four continuous-control tasks in simulated environments, and compare to diversity-based baselines.
Code (0)
등록된 구현이 없습니다.
Tasks
continuous-controlContinuous ControlDiversitySimilar Papers 제목 키워드 기반
Exploring the Manifold of Neural Networks Using Diffusion Geometry
Drawing motivation from the manifold hypothesis, which posits that most high-dimensional data lies on or near low-dimensional manifolds, we apply manifold learning to the space of neural networks. We learn manifolds wher…
Dimensionality ReductionHyperparameter OptimizationNeural Architecture SearchHeuristic Framework for Multi-Scale Testing of the Multi-Manifold Hypothesis
When analyzing empirical data, we often find that global linear models overestimate the number of parameters required. In such cases, we may ask whether the data lies on or near a manifold or a set of manifolds (a so-cal…
Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions
Denoising Diffusion Probabilistic Models (DDPM) are powerful state-of-the-art methods used to generate synthetic data from high-dimensional data distributions and are widely used for image, audio, and video generation as…
DenoisingGaussian ProcessesVideo GenerationDeepAtlas: a tool for effective manifold learning
Manifold learning builds on the "manifold hypothesis," which posits that data in high-dimensional datasets are drawn from lower-dimensional manifolds. Current tools generate global embeddings of data, rather than the loc…
Verifying the Union of Manifolds Hypothesis for Image Data
Deep learning has had tremendous success at learning low-dimensional representations of high-dimensional data. This success would be impossible if there was no hidden low-dimensional structure in data of interest; this e…
Inductive Bias