Fat-Tailed Variational Inference with Anisotropic Tail Adaptive Flows

2022-05-16 · Feynman Liang, Liam Hodgkinson, Michael W. Mahoney

While fat-tailed densities commonly arise as posterior and marginal distributions in robust models and scale mixtures, they present challenges when Gaussian-based variational inference fails to capture tail decay accurately. We first improve previous theory on tails of Lipschitz flows by quantifying how the tails affect the rate of tail decay and by expanding the theory to non-Lipschitz polynomial flows. Then, we develop an alternative theory for multivariate tail parameters which is sensitive to tail-anisotropy. In doing so, we unveil a fundamental problem which plagues many existing flow-based methods: they can only model tail-isotropic distributions (i.e., distributions having the same tail parameter in every direction). To mitigate this and enable modeling of tail-anisotropic targets, we propose anisotropic tail-adaptive flows (ATAF). Experimental results on both synthetic and real-world targets confirm that ATAF is competitive with prior work while also exhibiting appropriate tail-anisotropy.

📄 PDF Abstract BibTeX arXiv:2205.07918

Code (0)

등록된 구현이 없습니다.

Tasks

Variational Inference

Methods 이 논문이 사용한 방법론

Variational Inference 설명 없음

Posterior and variational inference for deep neural networks with heavy-tailed weights

2024-06-05 · Ismaël Castillo, Paul Egels

We consider deep neural networks in a Bayesian framework with a prior distribution sampling the network weights at random. Following a recent idea of Agapiou and Castillo (2023), who show that heavy-tailed prior distribu…

Model Selection Variational Inference

A variational Bayes framework for sparse adaptive estimation

2014-01-13 · Konstantinos E. Themelis, Athanasios A. Rontogiannis, Konstantinos D. Koutroumbas

Recently, a number of mostly $\ell_1$-norm regularized least squares type deterministic algorithms have been proposed to address the problem of \emph{sparse} adaptive signal estimation and system identification. From a B…

A Filtering Approach to Stochastic Variational Inference

2014-12-01 · NeurIPS 2014 12 · Neil Houlsby, David Blei

Stochastic variational inference (SVI) uses stochastic optimization to scale up Bayesian computation to massive data. We present an alternative perspective on SVI as approximate parallel coordinate ascent. SVI trades-off…

Stochastic Optimization Variational Inference

Tutorial and Survey on Probabilistic Graphical Model and Variational Inference in Deep Reinforcement Learning

2019-08-25 · Xudong Sun, Bernd Bischl

Aiming at a comprehensive and concise tutorial survey, recap of variational inference and reinforcement learning with Probabilistic Graphical Models are given with detailed derivations. Reviews and comparisons on recent …

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+2

ViRN: Variational Inference and Distribution Trilateration for Long-Tailed Continual Representation Learning

2025-07-23 · Hao Dai, Chong Tang, Jagmohan Chauhan arxiv

Continual learning (CL) with long-tailed data distributions remains a critical challenge for real-world AI systems, where models must sequentially adapt to new classes while retaining knowledge of old ones, despite sever…

Representation Learning Continual Learning