An Efficient Transformer Decoder with Compressed Sub-layers

2021-01-03 · Yanyang Li, Ye Lin, Tong Xiao, Jingbo Zhu

The large attention-based encoder-decoder network (Transformer) has become prevailing recently due to its effectiveness. But the high computation complexity of its decoder raises the inefficiency issue. By examining the mathematic formulation of the decoder, we show that under some mild conditions, the architecture could be simplified by compressing its sub-layers, the basic building block of Transformer, and achieves a higher parallelism. We thereby propose Compressed Attention Network, whose decoder layer consists of only one sub-layer instead of three. Extensive experiments on 14 WMT machine translation tasks show that our model is 1.42x faster with performance on par with a strong baseline. This strong baseline is already 2x faster than the widely used standard baseline without loss in performance.

📄 PDF Abstract BibTeX arXiv:2101.00542

Code (0)

등록된 구현이 없습니다.

Tasks

Decoder Machine Translation Translation

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Position-Wise Feed-Forward Layer 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Probing Word Translations in the Transformer and Trading Decoder for Encoder Layers

2020-03-21 · NAACL 2021 4 · Hongfei Xu, Josef van Genabith, Qiuhui Liu, Deyi Xiong

Due to its effectiveness and performance, the Transformer translation model has attracted wide attention, most recently in terms of probing-based approaches. Previous work focuses on using or probing source linguistic fe…

Decoder Translation Word Translation

DEED: Dynamic Early Exit on Decoder for Accelerating Encoder-Decoder Transformer Models

2023-11-15 · Peng Tang, Pengkai Zhu, Tian Li, Srikar Appalaraju 외

Encoder-decoder transformer models have achieved great success on various vision-language (VL) tasks, but they suffer from high inference latency. Typically, the decoder takes up most of the latency because of the auto-r…

Decoder

Balancing Cost and Benefit with Tied-Multi Transformers

2020-02-20 · WS 2020 7 · Raj Dabre, Raphael Rubino, Atsushi Fujita

We propose and evaluate a novel procedure for training multiple Transformers with tied parameters which compresses multiple models into one enabling the dynamic choice of the number of encoder and decoder layers during d…

Decoder Knowledge Distillation Machine Translation Model Compression+1

DecoderLens: Layerwise Interpretation of Encoder-Decoder Transformers

2023-10-05 · Anna Langedijk, Hosein Mohebbi, Gabriele Sarti, Willem Zuidema 외

In recent years, many interpretability methods have been proposed to help interpret the internal states of Transformer-models, at different levels of precision and complexity. Here, to analyze encoder-decoder Transformer…

Decoder Logical Reasoning Machine Translation Question Answering+2

GTrans: Grouping and Fusing Transformer Layers for Neural Machine Translation

2022-07-29 · Jian Yang, Yuwei Yin, Liqun Yang, Shuming Ma 외

Transformer structure, stacked by a sequence of encoder and decoder network layers, achieves significant development in neural machine translation. However, vanilla Transformer mainly exploits the top-layer representatio…