MEMe: An Accurate Maximum Entropy Method for Efficient Approximations in Large-Scale Machine Learning
Efficient approximation lies at the heart of large-scale machine learning problems. In this paper, we propose a novel, robust maximum entropy algorithm, which is capable of dealing with hundreds of moments and allows for computationally efficient approximations. We showcase the usefulness of the proposed method, its equivalence to constrained Bayesian variational inference and demonstrate its superiority over existing approaches in two applications, namely, fast log determinant estimation and information-theoretic Bayesian optimisation.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian OptimisationBIG-bench Machine LearningVariational InferenceSimilar Papers 제목 키워드 기반
Communicating via Markov Decision Processes
We consider the problem of communicating exogenous information by means of Markov decision process trajectories. This setting, which we call a Markov coding game (MCG), generalizes both source coding and a large class of…
Multi-agent Reinforcement LearningSample-Efficient Reinforcement Learning with Maximum Entropy Mellowmax Episodic Control
Deep networks have enabled reinforcement learning to scale to more complex and challenging domains, but these methods typically require large quantities of training data. An alternative is to use sample-efficient episodi…
Atari Gamesreinforcement-learningReinforcement LearningReinforcement Learning (RL)Multiscale Sparse Microcanonical Models
We study approximations of non-Gaussian stationary processes having long range correlations with microcanonical models. These models are conditioned by the empirical value of an energy vector, evaluated on a single reali…
Gaussian ProcessesPoint ProcessesTexture SynthesisPredictive Entropy Search for Efficient Global Optimization of Black-box Functions
We propose a novel information-theoretic approach for Bayesian optimization called Predictive Entropy Search (PES). At each iteration, PES selects the next evaluation point that maximizes the expected information gained …
Bayesian Optimizationglobal-optimizationImproving the Accuracy of Principal Component Analysis by the Maximum Entropy Method
Classical Principal Component Analysis (PCA) approximates data in terms of projections on a small number of orthogonal vectors. There are simple procedures to efficiently compute various functions of the data from the PC…