Improved Bayesian Compression

2017-11-17 · Marco Federici, Karen Ullrich, Max Welling

Compression of Neural Networks (NN) has become a highly studied topic in recent years. The main reason for this is the demand for industrial scale usage of NNs such as deploying them on mobile devices, storing them efficiently, transmitting them via band-limited channels and most importantly doing inference at scale. In this work, we propose to join the Soft-Weight Sharing and Variational Dropout approaches that show strong results to define a new state-of-the-art in terms of model compression.

📄 PDF Abstract BibTeX arXiv:1711.06494

Code (0)

등록된 구현이 없습니다.

Tasks

Model Compression

Methods 이 논문이 사용한 방법론

Variational Dropout Variational Dropout is a regularization technique based on dropout, but uses a variational inference grounded approach. In…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Data Compression with Bayesian Attention Networks

2021-03-04 · Anonymous

The lossless data compression algorithm based on Bayesian Attention Networks is derived from first principles.

Data Compression

On Compression Principle and Bayesian Optimization for Neural Networks

2020-06-23 · Michael Tetelman

Finding methods for making generalizable predictions is a fundamental problem of machine learning. By looking into similarities between the prediction problem for unknown data and the lossless compression we have found a…

Bayesian Optimization Dimensionality Reduction

QLSD: Quantised Langevin stochastic dynamics for Bayesian federated learning

2021-06-01 · Maxime Vono, Vincent Plassier, Alain Durmus, Aymeric Dieuleveut 외

The objective of Federated Learning (FL) is to perform statistical inference for data which are decentralised and stored locally on networked clients. FL raises many constraints which include privacy and data ownership, …

Federated Learning

Enhanced Bayesian Compression via Deep Reinforcement Learning

2019-06-01 · CVPR 2019 6 · Xin Yuan, Liangliang Ren, Jiwen Lu, Jie Zhou

In this paper, we propose an Enhanced Bayesian Compression method to flexibly compress the deep networks via reinforcement learning. Unlike the existing Bayesian compression method which cannot explicitly enforce quantiz…

Deep Reinforcement Learning Quantization reinforcement-learning Reinforcement Learning+1

A Bayesian Optimization Framework for Neural Network Compression

2019-10-01 · ICCV 2019 10 · Xingchen Ma, Amal Rannen Triki, Maxim Berman, Christos Sagonas 외

Neural network compression is an important step for deploying neural networks where speed is of high importance, or on devices with limited memory. It is necessary to tune compression parameters in order to achieve the d…

Bayesian Optimization Knowledge Distillation Neural Network Compression