홈 › Papers

Simple, Distributed, and Accelerated Probabilistic Programming

2018-11-05 · NeurIPS 2018 12 · Dustin Tran, Matthew Hoffman, Dave Moore, Christopher Suter, Srinivas Vasudevan, Alexey Radul, Matthew Johnson, Rif A. Saurous

We describe a simple, low-level approach for embedding probabilistic programming in a deep learning ecosystem. In particular, we distill probabilistic programming down to a single abstraction---the random variable. Our lightweight implementation in TensorFlow enables numerous applications: a model-parallel variational auto-encoder (VAE) with 2nd-generation tensor processing units (TPUv2s); a data-parallel autoregressive model (Image Transformer) with TPUv2s; and multi-GPU No-U-Turn Sampler (NUTS). For both a state-of-the-art VAE on 64x64 ImageNet and Image Transformer on 256x256 CelebA-HQ, our approach achieves an optimal linear speedup from 1 to 256 TPUv2 chips. With NUTS, we see a 100x speedup on GPUs over Stan and 37x over PyMC3.

📄 PDF Abstract BibTeX arXiv:1811.02091

Code (1)

google/edward2 tf

Tasks

GPUProbabilistic Programming

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Position-Wise Feed-Forward Layer 설명 없음

Residual Connection 설명 없음

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

ReLU How Do I Communicate to Expedia? How Do I Communicate to Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Live Support & Special Travel…

Pyro: Deep Universal Probabilistic Programming

2018-10-18 · Eli Bingham, Jonathan P. Chen, Martin Jankowiak, Fritz Obermeyer 외

Pyro is a probabilistic programming language built on Python as a platform for developing advanced probabilistic models in AI research. To scale to large datasets and high-dimensional models, Pyro uses stochastic variati…

GPU Probabilistic Programming Variational Inference

Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro

2019-12-24 · Du Phan, Neeraj Pradhan, Martin Jankowiak

NumPyro is a lightweight library that provides an alternate NumPy backend to the Pyro probabilistic programming language with the same modeling interface, language primitives and effect handling abstractions. Effect hand…

Probabilistic Programming

Spreadsheet Probabilistic Programming

2016-06-14 · Mike Wu, Yura Perov, Frank Wood, Hongseok Yang

Spreadsheet workbook contents are simple programs. Because of this, probabilistic programming techniques can be used to perform Bayesian inversion of spreadsheet computations. What is more, existing execution engines in …

Decision Making Decision Making Under Uncertainty Probabilistic Programming Variational Inference

A Low-Level Index for Distributed Logic Programming

2020-09-22 · Thomas Prokosch

A distributed logic programming language with support for meta-programming and stream processing offers a variety of interesting research problems, such as: How can a versatile and stable data structure for the indexing …

Accelerated first-order primal-dual proximal methods for linearly constrained composite convex programming

2016-06-29 · Yangyang Xu

Motivated by big data applications, first-order methods have been extremely popular in recent years. However, naive gradient methods generally converge slowly. Hence, much efforts have been made to accelerate various fir…

Denoising Image Denoising