Sandwich Transformer

2000년 도입 · 논문 2편에서 사용

A Sandwich Transformer is a variant of a Transformer that reorders sublayers in the architecture to achieve better performance. The reordering is based on the authors' analysis that models with more self-attention toward the bottom and more feedforward sublayers toward the top tend to perform better in general.

출처: Improving Transformer Models by Reordering their Sublayers

소개 논문: Improving Transformer Models by Reordering their Sublayers