VISA: Variational Inference with Sequential Sample-Average Approximations
We present variational inference with sequential sample-average approximation (VISA), a method for approximate inference in computationally intensive models, such as those based on numerical simulations. VISA extends importance-weighted forward-KL variational inference by employing a sequence of sample-average approximations, which are considered valid inside a trust region. This makes it possible to reuse model evaluations across multiple gradient steps, thereby reducing computational cost. We perform experiments on high-dimensional Gaussians, Lotka-Volterra dynamics, and a Pickover attractor, which demonstrate that VISA can achieve comparable approximation accuracy to standard importance-weighted forward-KL variational inference with computational savings of a factor two or more for conservatively chosen learning rates.
Code (0)
등록된 구현이 없습니다.
Tasks
validVariational InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
CADIC: Continual Anomaly Detection Based on Incremental Coreset
The primary objective of Continual Anomaly Detection (CAD) is to learn the normal patterns of new tasks under dynamic data distribution assumptions while mitigating catastrophic forgetting. Existing embedding-based CAD a…
Anomaly DetectionSequential Function-Space Variational Inference via Gaussian Mixture Approximation
Continual learning is learning from a sequence of tasks with the aim of learning new tasks without forgetting old tasks. Sequential function-space variational inference (SFSVI) is a continual learning method based on var…
Continual LearningVariational InferenceViRN: Variational Inference and Distribution Trilateration for Long-Tailed Continual Representation Learning
Continual learning (CL) with long-tailed data distributions remains a critical challenge for real-world AI systems, where models must sequentially adapt to new classes while retaining knowledge of old ones, despite sever…
Representation LearningContinual LearningVariational Dynamic Mixtures
Deep probabilistic time series forecasting models have become an integral part of machine learning. While several powerful generative models have been proposed, we provide evidence that their associated inference models …
Probabilistic Time Series ForecastingTime SeriesTime Series AnalysisTime Series ForecastingDiffusion ReRoll: Revisable Denoising for Robotic Sequential Prediction
We propose Diffusion ReRoll, a diffusion-based framework for robotic sequential prediction that enables revisable denoising over horizons. Existing diffusion-based sequence predictors typically perform a single monotonic…