DeepViT

2000년 도입 · 논문 2편에서 사용

DeepViT is a type of vision transformer that replaces the self-attention layer within the transformer block with a Re-attention module to address the issue of attention collapse and enables training deeper ViTs.