UL2

2000년 도입 · 논문 8편에서 사용

UL2 is a unified framework for pretraining models that are universally effective across datasets and setups. UL2 uses Mixture-of-Denoisers (MoD), a pre-training objective that combines diverse pre-training paradigms together. UL2 introduces a notion of mode switching, wherein downstream fine-tuning is associated with specific pre-training schemes.

출처: UL2: Unifying Language Learning Paradigms

소개 논문: UL2: Unifying Language Learning Paradigms