Mitigating mode collapse in normalizing flows by annealing with an adaptive schedule: Application to parameter estimation
Normalizing flows (NFs) provide uncorrelated samples from complex distributions, making them an appealing tool for parameter estimation. However, the practical utility of NFs remains limited by their tendency to collapse to a single mode of a multimodal distribution. In this study, we show that annealing with an adaptive schedule based on the effective sample size (ESS) can mitigate mode collapse. We demonstrate that our approach can converge the marginal likelihood for a biochemical oscillator model fit to time-series data in ten-fold less computation time than a widely used ensemble Markov chain Monte Carlo (MCMC) method. We show that the ESS can also be used to reduce variance by pruning the samples. We expect these developments to be of general use for sampling with NFs and discuss potential opportunities for further improvements.
Code (0)
등록된 구현이 없습니다.
Tasks
parameter estimationTime SeriesMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Annealing in variational inference mitigates mode collapse: A theoretical study on Gaussian mixtures
Mode collapse, the failure to capture one or more modes when targetting a multimodal distribution, is a central challenge in modern variational inference. In this work, we provide a mathematical analysis of annealing bas…
FlowVAT: Normalizing Flow Variational Inference with Affine-Invariant Tempering
Multi-modal and high-dimensional posteriors present significant challenges for variational inference, causing mode-seeking behavior and collapse despite the theoretical expressiveness of normalizing flows. Traditional an…
Variational InferenceDetecting and Mitigating Mode-Collapse for Flow-based Sampling of Lattice Field Theories
We study the consequences of mode-collapse of normalizing flows in the context of lattice field theory. Normalizing flows allow for independent sampling. For this reason, it is hoped that they can avoid the tunneling pro…
Boosting Summarization with Normalizing Flows and Aggressive Training
This paper presents FlowSUM, a normalizing flows-based variational encoder-decoder framework for Transformer-based summarization. Our approach tackles two primary challenges in variational summarization: insufficient sem…
DecoderKnowledge DistillationText SummarizationAdaAnn: Adaptive Annealing Scheduler for Probability Density Approximation
Approximating probability distributions can be a challenging task, particularly when they are supported over regions of high geometrical complexity or exhibit multiple modes. Annealing can be used to facilitate this task…
Computational Efficiencyparameter estimationVariational Inference