Embracing the chaos: analysis and diagnosis of numerical instability in variational flows
In this paper, we investigate the impact of numerical instability on the reliability of sampling, density evaluation, and evidence lower bound (ELBO) estimation in variational flows. We first empirically demonstrate that common flows can exhibit a catastrophic accumulation of error: the numerical flow map deviates significantly from the exact map -- which affects sampling -- and the numerical inverse flow map does not accurately recover the initial input -- which affects density and ELBO computations. Surprisingly though, we find that results produced by flows are often accurate enough for applications despite the presence of serious numerical instability. In this work, we treat variational flows as dynamical systems, and leverage shadowing theory to elucidate this behavior via theoretical guarantees on the error of sampling, density evaluation, and ELBO estimation. Finally, we develop and empirically test a diagnostic procedure that can be used to validate results produced by numerically unstable flows in practice.
Code (0)
등록된 구현이 없습니다.
Tasks
DiagnosticSimilar Papers 제목 키워드 기반
Sparse chaos in cortical circuits
Nerve impulses, the currency of information flow in the brain, are generated by an instability of the neuronal membrane potential dynamics. Neuronal circuits exhibit collective chaos that appears essential for learning, …
Numerical Instability and Chaos: Quantifying the Unpredictability of Large Language Models
As Large Language Models (LLMs) are increasingly integrated into agentic workflows, their unpredictability stemming from numerical instability has emerged as a critical reliability issue. While recent studies have demons…
Machine learning for prediction of extreme statistics in modulation instability
A central area of research in nonlinear science is the study of instabilities that drive the emergence of extreme events. Unfortunately, experimental techniques for measuring such phenomena often provide only partial cha…
BIG-bench Machine LearningMulti-agent Performative Prediction: From Global Stability and Optimality to Chaos
The recent framework of performative prediction is aimed at capturing settings where predictions influence the target/outcome they want to predict. In this paper, we introduce a natural multi-agent version of this framew…
PIPPS: Flexible Model-Based Policy Search Robust to the Curse of Chaos
Previously, the exploding gradient problem has been explained to be central in deep learning and model-based reinforcement learning, because it causes numerical issues and instability in optimization. Our experiments in …
Model-based Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)