Efficient Approximate Inference with Walsh-Hadamard Variational Inference
Variational inference offers scalable and flexible tools to tackle intractable Bayesian inference of modern statistical models like Bayesian neural networks and Gaussian processes. For largely over-parameterized models, however, the over-regularization property of the variational objective makes the application of variational inference challenging. Inspired by the literature on kernel methods, and in particular on structured approximations of distributions of random matrices, this paper proposes Walsh-Hadamard Variational Inference, which uses Walsh-Hadamard-based factorization strategies to reduce model parameterization, accelerate computations, and increase the expressiveness of the approximate posterior beyond fully factorized ones.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian InferenceGaussian ProcessesVariational InferenceSimilar Papers 제목 키워드 기반
Walsh-Hadamard Variational Inference for Bayesian Deep Learning
Over-parameterized models, such as DeepNets and ConvNets, form a class of models that are routinely adopted in a wide variety of applications, and for which Bayesian inference is desirable but extremely challenging. Vari…
Bayesian InferenceDeep LearningVariational InferenceMcKernel: A Library for Approximate Kernel Expansions in Log-linear Time
Kernel Methods Next Generation (KMNG) introduces a framework to use kernel approximates in the mini-batch setting with SGD Optimizer as an alternative to Deep Learning. McKernel is a C++ library for KMNG ML Large-scale. …
CPUGeneral ClassificationFast Walsh-Hadamard Transform and Smooth-Thresholding Based Binary Layers in Deep Neural Networks
In this paper, we propose a novel layer based on fast Walsh-Hadamard transform (WHT) and smooth-thresholding to replace $1\times 1$ convolution layers in deep neural networks. In the WHT domain, we denoise the transform …
Multiallelic Walsh transforms
A closed formula multiallelic Walsh (or Hadamard) transform is introduced. Basic results are derived, and a statistical interpretation of some of the resulting linear forms is discussed.
Walsh-Hadamard Neural Operators for Solving PDEs with Discontinuous Coefficients
Neural operators have emerged as powerful tools for learning solution operators of partial differential equations (PDEs). However, standard spectral methods based on Fourier transforms struggle with problems involving di…