Transformer with Memory Replay

2022-05-19 · Rui Liu, Barzan Mozafari

Transformers achieve state-of-the-art performance for natural language processing tasks by pre-training on large-scale text corpora. They are extremely compute-intensive and have very high sample complexity. Memory replay is a mechanism that remembers and reuses past examples by saving to and replaying from a memory buffer. It has been successfully used in reinforcement learning and GANs due to better sample efficiency. In this paper, we propose \emph{Transformer with Memory Replay} (TMR), which integrates memory replay with transformer, making transformer more sample-efficient. Experiments on GLUE and SQuAD benchmark datasets show that Transformer with Memory Replay achieves at least $1\%$ point increase compared to the baseline transformer model when pretrained with the same number of examples. Further, by adopting a careful design that reduces the wall-clock time overhead of memory replay, we also empirically achieve a better runtime efficiency.

📄 PDF Abstract BibTeX arXiv:2205.09869

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Experience Replay Addresses Loss of Plasticity in Continual Learning

2025-03-25 · Jiuqi Wang, Rohan Chandra, Shangtong Zhang

Loss of plasticity is one of the main challenges in continual learning with deep neural networks, where neural networks trained via backpropagation gradually lose their ability to adapt to new tasks and perform significa…

Continual Learning In-Context Learning

RMAAT: Astrocyte-Inspired Memory Compression and Replay for Efficient Long-Context Transformers

2026-01-01 · Md Zesun Ahmed Mia, Malyaban Bal, Abhronil Sengupta arxiv

The quadratic complexity of self-attention mechanism presents a significant impediment to applying Transformer models to long sequences. This work explores computational principles derived from astrocytes-glial cells cri…

Replay Memory as An Empirical MDP: Combining Conservative Estimation with Experience Replay

2023-05-01 · ICLR 2023 5 · Hongming Zhang, Chenjun Xiao, Han Wang, Jun Jin 외

Experience replay, which stores transitions in a replay memory for repeated use, plays an important role of improving sample efficiency in reinforcement learning. Existing techniques such as reweighted sampling, episodic…

GEAR: A GPU-Centric Experience Replay System for Large Reinforcement Learning Models

2023-10-08 · Hanjing Wang, Man-Kit Sit, Congjie He, Ying Wen 외

This paper introduces a distributed, GPU-centric experience replay system, GEAR, designed to perform scalable reinforcement learning (RL) with large sequence models (such as transformers). With such models, existing syst…

GPU Reinforcement Learning (RL)

Memformer: A Memory-Augmented Transformer for Sequence Modeling

2020-10-14 · Qingyang Wu, Zhenzhong Lan, Kun Qian, Jing Gu 외

Transformers have reached remarkable success in sequence modeling. However, these models have efficiency issues as they need to store all the history token-level representations as memory. We present Memformer, an effici…

Efficient Neural Network Language Modelling