Stabilizing the Kumaraswamy Distribution
Large-scale latent variable models require expressive continuous distributions that support efficient sampling and low-variance differentiation, achievable through the reparameterization trick. The Kumaraswamy (KS) distribution is both expressive and supports the reparameterization trick with a simple closed-form inverse CDF. Yet, its adoption remains limited. We identify and resolve numerical instabilities in the inverse CDF and log-pdf, exposing issues in libraries like PyTorch and TensorFlow. We then introduce simple and scalable latent variable models based on the KS, improving exploration-exploitation trade-offs in contextual multi-armed bandits and enhancing uncertainty quantification for link prediction with graph neural networks. Our results support the stabilized KS distribution as a core component in scalable variational models for bounded latent variables.
Code (0)
등록된 구현이 없습니다.
Tasks
Link PredictionMulti-Armed BanditsUncertainty QuantificationSimilar Papers 제목 키워드 기반
Improving neural networks with bunches of neurons modeled by Kumaraswamy units: Preliminary study
Deep neural networks have recently achieved state-of-the-art results in many machine learning problems, e.g., speech recognition or object recognition. Hitherto, work on rectified linear units (ReLU) provides empirical a…
Object Recognitionspeech-recognitionSpeech RecognitionA New Distribution on the Simplex with Auto-Encoding Applications
We construct a new distribution for the simplex using the Kumaraswamy distribution and an ordered stick-breaking process. We explore and develop the theoretical properties of this new distribution and prove that it exhib…
Learnt Sparsification for Interpretable Graph Neural Networks
Graph neural networks (GNNs) have achieved great success on various tasks and fields that require relational modeling. GNNs aggregate node features using the graph structure as inductive biases resulting in flexible and …
Pareto Smoothed Importance Sampling
Importance weighting is a general way to adjust Monte Carlo integration to account for draws from the wrong distribution, but the resulting estimate can be highly variable when the importance ratios have a heavy right ta…
DiagnosticA Minimum Variance Path Principle for Accurate and Stable Score-Based Density Ratio Estimation
Score-based methods are powerful across machine learning, but they face a paradox: theoretically path-independent, yet practically path-dependent. We resolve this by proving that practical training objectives differ from…