SimVLM

2000년 도입 · 논문 3편에서 사용

SimVLM is a minimalist pretraining framework to reduce training complexity by exploiting large-scale weak supervision. It is trained end-to-end with a single prefix language modeling (PrefixLM) objective. PrefixLM enables bidirectional attention within the prefix sequence, and thus it is applicable for both decoder-only and encoder-decoder sequence-to-sequence language models.

출처: SimVLM: Simple Visual Language Model Pretraining with Weak Supervision

소개 논문: SimVLM: Simple Visual Language Model Pretraining with Weak Supervision