Compression with Flows via Local Bits-Back Coding
Likelihood-based generative models are the backbones of lossless compression due to the guaranteed existence of codes with lengths close to negative log likelihood. However, there is no guaranteed existence of computationally efficient codes that achieve these lengths, and coding algorithms must be hand-tailored to specific types of generative models to ensure computational efficiency. Such coding algorithms are known for autoregressive models and variational autoencoders, but not for general types of flow models. To fill in this gap, we introduce local bits-back coding, a new compression technique for flow models. We present efficient algorithms that instantiate our technique for many popular types of flows, and we demonstrate that our algorithms closely achieve theoretical codelengths for state-of-the-art flow models on high-dimensional data.
Code (1)
Tasks
Computational EfficiencySimilar Papers 제목 키워드 기반
Split Hierarchical Variational Compression
Variational autoencoders (VAEs) have witnessed great success in performing the compression of image datasets. This success, made possible by the bits-back coding framework, has produced competitive compression performanc…
Image CompressionLossless Compression using Continuously-Indexed Normalizing Flows
Recently, a class of deep generative models known as continuously-indexed flows (CIFs) have expanding the modelling capacity of normalizing flows (NFs) in the context of both density estimation and variational inference.…
Density EstimationVariational InferenceImproving Lossless Compression Rates via Monte Carlo Bits-Back Coding
Latent variable models have been successfully applied in lossless compression with the bits-back coding algorithm. However, bits-back suffers from an increase in the bitrate equal to the KL divergence between the approxi…
Data CompressionEnd-to-End Image Compression with Segmentation Guided Dual Coding for Wind Turbines
Transferring large volumes of high-resolution images during wind turbine inspections introduces a bottleneck in assessing and detecting severe defects. Efficient coding must preserve high fidelity in blade regions while …
Image CompressionBit-Swap: Recursive Bits-Back Coding for Lossless Compression with Hierarchical Latent Variables
The bits-back argument suggests that latent variable models can be turned into lossless compression schemes. Translating the bits-back argument into efficient and practical lossless compression schemes for general latent…