Network Pruning for Low-Rank Binary Index
Pruning is an efficient model compression technique to remove redundancy in the connectivity of deep neural networks (DNNs). A critical problem to represent sparse matrices after pruning is that if fewer bits are used for quantization and pruning rate is enhanced, then the amount of index becomes relatively larger. Moreover, an irregular index form leads to low parallelism for convolutions and matrix multiplications. In this paper, we propose a new network pruning technique that generates a low-rank binary index matrix to compress index data significantly. Specifically, the proposed compression method finds a particular fine-grained pruning mask that can be decomposed into two binary matrices while decompressing index data is performed by simple binary matrix multiplication. We also propose a tile-based factorization technique that not only lowers memory requirements but also enhances compression ratio. Various DNN models (including conv layers and LSTM layers) can be pruned with much fewer indices compared to previous sparse matrix formats while maintaining the same pruning rate.
Code (0)
등록된 구현이 없습니다.
Tasks
Model CompressionNetwork PruningQuantizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Network Pruning for Low-Rank Binary Indexing
Pruning is an efficient model compression technique to remove redundancy in the connectivity of deep neural networks (DNNs). Computations using sparse matrices obtained by pruning parameters, however, exhibit vastly diff…
Model CompressionNetwork PruningPruning the Index Contents for Memory Efficient Open-Domain QA
This work presents a novel pipeline that demonstrates what is achievable with a combined effort of state-of-the-art approaches. Specifically, it proposes the novel R2-D2 (Rank twice, reaD twice) pipeline composed of retr…
Open-Domain Question AnsweringA Static Pruning Study on Sparse Neural Retrievers
Sparse neural retrievers, such as DeepImpact, uniCOIL and SPLADE, have been introduced recently as an efficient and effective way to perform retrieval with inverted indexes. They aim to learn term importance and, in some…
Document RankingRetrievalStatic Pruning Across Sparse Retrieval Regimes: What Transfers, What Breaks, and What Still Helps
Static pruning is widely used to accelerate sparse neural retrieval, yet existing studies each validate their conclusions within a single custom pipeline, leaving it unclear which findings transfer to modern engines with…
Natural QuestionsEfficient Neural Ranking using Forward Indexes
Neural document ranking approaches, specifically transformer models, have achieved impressive gains in ranking performance. However, query processing using such over-parameterized models is both resource and time intensi…
CPUDocument RankingRetrievalSemantic Similarity+1