RAG

2000년 도입 · 논문 1,286편에서 사용

Retriever-Augmented Generation, or RAG, is a type of language generation model that combines pre-trained parametric and non-parametric memory for language generation. Specifically, the parametric memory is a pre-trained seq2seq model and the non-parametric memory is a dense vector index of Wikipedia, accessed with a pre-trained neural retriever. For query $x$, Maximum Inner Product Search (MIPS) is used to find the top-K documents $z\_{i}$. For final prediction $y$, we treat $z$ as a latent variable and marginalize over seq2seq predictions given different documents.

출처: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

소개 논문: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks