GMVAE

2000년 도입 · 논문 5편에서 사용

GMVAE, or Gaussian Mixture Variational Autoencoder, is a stochastic regularization layer for transformers. A GMVAE layer is trained using a 700-dimensional internal representation of the first MLP layer. For every output from the first MLP layer, the GMVAE layer first computes a latent low-dimensional representation sampling from the GMVAE posterior distribution to then provide at the output a reconstruction sampled from a generative model.

출처: Regularizing Transformers With Deep Probabilistic Layers

소개 논문: Regularizing Transformers With Deep Probabilistic Layers