GPT

2000년 도입 · 논문 1,212편에서 사용

GPT is a Transformer-based architecture and training procedure for natural language processing tasks. Training follows a two-stage procedure. First, a language modeling objective is used on the unlabeled data to learn the initial parameters of a neural network model. Subsequently, these parameters are adapted to a target task using the corresponding supervised objective.

출처: Improving Language Understanding by Generative Pre-Training

소개 논문: Improving Language Understanding by Generative Pre-Training