CULL-MT: Compression Using Language and Layer pruning for Machine Translation
Multilingual machine translation models often outperform traditional bilingual models by leveraging translation knowledge transfer. Recent advancements have led to these models supporting hundreds of languages and achieving state-of-the-art results across various translation directions. However, as these models grow larger, their inference operations become increasingly costly. In many use cases, there is no need to support such a wide range of language pairs, as translation is typically needed in only a few selected directions. In this paper, we present CULL-MT, a compression method for machine translation models based on structural layer pruning and selected language directions. Our approach identifies and prunes unimportant layers using a greedy strategy, then mitigates the impact by applying knowledge distillation from the original model along with parameter-efficient fine-tuning. We apply CULL-MT to the NLLB-3.3B and LLaMA3.1-8B-Instruct models. In a multi-way translation scenario (Persian, French, and German to English), we find the NLLB-3.3B model to be robust, allowing 25% of layers to be pruned with only a 0.9 spBLEU drop. However, LLaMA3.1-8B-Instruct is more sensitive, with a 2.0 spBLEU drop after pruning 5 layers.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge DistillationMachine Translationparameter-efficient fine-tuningTransfer LearningTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Iterative Layer Pruning for Efficient Translation Inference
Large language models (LLMs) have transformed many areas of natural language processing, including machine translation. However, efficient deployment of LLMs remains challenging due to their intensive computational requi…
Machine TranslationModel CompressionStructural Anchor Pruning: Training-Free Multi-Vector Compression for Visual Document Retrieval
Recent Vision-Language Models (e.g., ColPali) enable fine-grained Visual Document Retrieval (VDR) but incur prohibitive multi-vector index storage overhead. Existing training-free pruning methods either rely on heuristic…
Just CHOP: Embarrassingly Simple LLM Compression
Large language models (LLMs) enable unparalleled few- and zero-shot reasoning capabilities but at a high computational footprint. A growing assortment of methods for compression promises to reduce the computational burde…
Knowledge DistillationLanguage ModelingLanguage ModellingLarge Language Model+1Effective Layer Pruning Through Similarity Metric Perspective
Deep neural networks have been the predominant paradigm in machine learning for solving cognitive tasks. Such models, however, are restricted by a high computational overhead, limiting their applicability and hindering a…
Towards Optimal Compression: Joint Pruning and Quantization
Model compression is instrumental in optimizing deep neural network inference on resource-constrained hardware. The prevailing methods for network compression, namely quantization and pruning, have been shown to enhance …
Model CompressionNeural Architecture SearchQuantization