Fully Quantized Transformer for Machine Translation
State-of-the-art neural machine translation methods employ massive amounts of parameters. Drastically reducing computational costs of such methods without affecting performance has been up to this point unsuccessful. To this end, we propose FullyQT: an all-inclusive quantization strategy for the Transformer. To the best of our knowledge, we are the first to show that it is possible to avoid any loss in translation quality with a fully quantized Transformer. Indeed, compared to full-precision, our 8-bit models score greater or equal BLEU on most tasks. Comparing ourselves to all previously proposed methods, we achieve state-of-the-art quantization results.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationQuantizationTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
DiscreTalk: Text-to-Speech as a Machine Translation Problem
This paper proposes a new end-to-end text-to-speech (E2E-TTS) model based on neural machine translation (NMT). The proposed model consists of two components; a non-autoregressive vector quantized variational autoencoder …
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Language ModellingMachine Translation+6FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformer
Network quantization significantly reduces model inference complexity and has been widely used in real-world deployments. However, most existing quantization methods have been developed mainly on Convolutional Neural Net…
QuantizationRepresentation Collapse in Machine Translation Through the Lens of Angular Dispersion
Modern neural translation models based on the Transformer architecture are known for their high performance, particularly when trained on high-resource datasets. A standard next-token prediction training strategy, while …
Machine TranslationEfficient 8-Bit Quantization of Transformer Neural Machine Language Translation Model
In this work, we quantize a trained Transformer machine language translation model leveraging INT8/VNNI instructions in the latest Intel$^\circledR$ Xeon$^\circledR$ Cascade Lake processors to improve inference performan…
CPUQuantizationTranslationInducing Systematicity in Transformers by Attending to Structurally Quantized Embeddings
Transformers generalize to novel compositions of structures and entities after being trained on a complex dataset, but easily overfit on datasets of insufficient complexity. We observe that when the training set is suffi…
Machine TranslationQuantizationSemantic ParsingWord Embeddings