GPSA

2000년 도입 · 논문 3편에서 사용

Gated Positional Self-Attention (GPSA) is a self-attention module for vision transformers, used in the ConViT architecture, that can be initialized as a convolutional layer -- helping a ViT learn inductive biases about locality.

출처: ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases

소개 논문: ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases