LeViT Attention Block

2000년 도입 · 논문 2편에서 사용

LeViT Attention Block is a module used for attention in the LeViT architecture. Its main feature is providing positional information within each attention block, i.e. where we explicitly inject relative position information in the attention mechanism. This is achieved by adding an attention bias to the attention maps.

출처: LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference

소개 논문: LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference