Fastformer

2000년 도입 · 논문 4편에서 사용

Fastformer is an type of Transformer which uses additive attention as a building block. Instead of modeling the pair-wise interactions between tokens, additive attention is used to model global contexts, and then each token representation is further transformed based on its interaction with global context representations.

출처: Fastformer: Additive Attention Can Be All You Need

소개 논문: Fastformer: Additive Attention Can Be All You Need