On the Normalizing Constant of the Continuous Categorical Distribution
Probability distributions supported on the simplex enjoy a wide range of applications across statistics and machine learning. Recently, a novel family of such distributions has been discovered: the continuous categorical. This family enjoys remarkable mathematical simplicity; its density function resembles that of the Dirichlet distribution, but with a normalizing constant that can be written in closed form using elementary functions only. In spite of this mathematical simplicity, our understanding of the normalizing constant remains far from complete. In this work, we characterize the numerical behavior of the normalizing constant and we present theoretical and methodological advances that can, in turn, help to enable broader applications of the continuous categorical distribution. Our code is available at https://github.com/cunningham-lab/cb_and_cc/.
Code (2)
Similar Papers 제목 키워드 기반
Argmax Flows: Learning Categorical Distributions with Normalizing Flows
This paper introduces a new method to define and train continuous distributions such as normalizing flows directly on categorical data, for example text and image segmentation. The generative model is defined by a compos…
Image SegmentationSemantic SegmentationCategorical Normalizing Flows via Continuous Transformations
Despite their popularity, to date, the application of normalizing flows on categorical data stays limited. The current practice of using dequantization to map discrete data to a continuous space is inapplicable as catego…
DecoderInductive BiasVariational InferenceReliable Categorical Variational Inference with Mixture of Discrete Normalizing Flows
Variational approximations are increasingly based on gradient-based optimization of expectations estimated by sampling. Handling discrete latent variables is then challenging because the sampling process is not different…
validVariational InferenceArgmax Flows and Multinomial Diffusion: Learning Categorical Distributions
Generative flows and diffusion models have been predominantly trained on ordinal data, for example natural images. This paper introduces two extensions of flows and diffusion for categorical data such as language or imag…
DenoisingImage SegmentationLanguage ModellingSegmentation+1Maximum likelihood estimation of the Fisher-Bingham distribution via efficient calculation of its normalizing constant
This paper proposes an efficient numerical integration formula to compute the normalizing constant of Fisher--Bingham distributions. This formula uses a numerical integration formula with the continuous Euler transform t…
Numerical Integration