Lossless Compression of Neural Network Components: Weights, Checkpoints, and K/V Caches in Low-Precision Formats

2025-08-20 · Anat Heilper, Doron Singer arxiv

As deep learning models grow and deployment becomes more widespread, reducing the storage and transmission costs of neural network weights has become increasingly important. While prior work such as ZipNN has shown that lossless compression methods - particularly those based on Huffman encoding floating-point exponents can significantly reduce model sizes, these techniques have primarily been applied to higher-precision formats such as FP32 and BF16. In this work, we extend the ZipNN approach to lower-precision floating-point formats, specifically FP8 and FP4, which are gaining popularity for efficient inference. We design a compression method that separates and compresses the exponent and mantissa components independently using entropy coding. Our evaluation shows compression ratios up to 62% for BF16 and 83% for FP8. We also investigate the compressibility of key-value (K/V) cache tensors used in large language models (LLMs), finding that they, too, exhibit compressible patterns, enabling memory savings during deployment.

📄 PDF Abstract BibTeX arXiv:2508.19263

Code (0)

등록된 구현이 없습니다.

ZigZagkv: Dynamic KV Cache Compression for Long-context Modeling based on Layer Uncertainty

2024-12-12 · Meizhi Zhong, Xikai Liu, Chen Zhang, Yikun Lei 외

Large Language models (LLMs) have become a research hotspot. To accelerate the inference of LLMs, storing computed caches in memory has become the standard technique. However, as the inference length increases, growing K…

Homogeneous Keys, Heterogeneous Values: Exploiting Local KV Cache Asymmetry for Long-Context LLMs

2025-06-04 · Wanyun Cui, Mingwei Xu

Recent advances in Large Language Models (LLMs) have highlighted the critical importance of extending context length, yet the quadratic complexity of attention mechanisms poses significant challenges for efficient long-c…

Lossless Tensor Compression as Program Synthesis

2026-08-03 · Jieke Shi, Junda He, Wenjia Jiang, Weifeng Sun 외 hf

Model checkpoints are growing in both number and size, which makes archival, transfer, and deployment increasingly costly. General-purpose compressors can reduce storage requirements but ignore tensor structure, whereas …

Program Synthesis Image Generation

SplitZip: Ultra Fast Lossless KV Compression for Disaggregated LLM Serving

2026-05-03 · Yipin Guo, Siddharth Joshi arxiv

Contemporary systems serving large language models (LLMs) have adopted prefill-decode disaggregation to load-balance between the compute-bound prefill phase and the memory-bound decode phase. Under this design, prefill w…

ExCP: Extreme LLM Checkpoint Compression via Weight-Momentum Joint Shrinking

2024-06-17 · Wenshuo Li, Xinghao Chen, Han Shu, Yehui Tang 외

Large language models (LLM) have recently attracted significant attention in the field of artificial intelligence. However, the training process of these models poses significant challenges in terms of computational and …

Model Optimization Quantization