Trans-Encoder
2000년 도입 · 논문 1편에서 사용
Unsupervised knowledge distillation from a pretrained language model to *itself*, by alternating between its bi- and cross-encoder forms.
출처: Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations
소개 논문: Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations
Self-Supervised Learning · GeneralSentence Embeddings · Natural Language Processing