Thanos: A Block-wise Pruning Algorithm for Efficient Large Language Model Compression
This paper presents Thanos, a novel weight-pruning algorithm designed to reduce the memory footprint and enhance the computational efficiency of large language models (LLMs) by removing redundant weights while maintaining accuracy. Thanos introduces a block-wise pruning strategy with adaptive masks that dynamically adjust to weight importance, enabling flexible sparsity patterns and structured formats, such as $n:m$ sparsity, optimized for hardware acceleration. Experimental evaluations demonstrate that Thanos achieves state-of-the-art performance in structured pruning and outperforms existing methods in unstructured pruning. By providing an efficient and adaptable approach to model compression, Thanos offers a practical solution for deploying large models in resource-constrained environments.
Code (1)
Tasks
Computational EfficiencyLanguage ModelingLanguage ModellingLarge Language ModelModel CompressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Ethanos: Lightweight Bootstrapping for Ethereum
As ethereum blockchain has become popular, the number of users and transactions has skyrocketed, causing an explosive increase of its data size. As a result, ordinary clients using PCs or smartphones cannot easily bootst…
ABCP: Automatic Block-wise and Channel-wise Network Pruning via Joint Search
Currently, an increasing number of model pruning methods are proposed to resolve the contradictions between the computer powers required by the deep learning models and the resource-constrained devices. However, most of …
Deep Reinforcement LearningNetwork PruningRealizing Unaligned Block-wise Pruning for DNN Acceleration on Mobile Devices
With the recent proliferation of on-device AI, there is an increasing need to run computationally intensive DNNs directly on mobile devices. However, the limited computing and memory resources of these devices necessitat…
BESA: Pruning Large Language Models with Blockwise Parameter-Efficient Sparsity Allocation
Large language models (LLMs) have demonstrated outstanding performance in various tasks, such as text summarization, text question-answering, and etc. While their performance is impressive, the computational footprint du…
GPUQuestion AnsweringText SummarizationScalable iterative pruning of large language and vision models using block coordinate descent
Pruning neural networks, which involves removing a fraction of their weights, can often maintain high accuracy while significantly reducing model complexity, at least up to a certain limit. We present a neural network pr…
Combinatorial OptimizationNetwork Pruning