Charformer

2000년 도입 · 논문 5편에서 사용

Charformer is a type of Transformer model that learns a subword tokenization end-to-end as part of the model. Specifically it uses GBST that automatically learns latent subword representations from characters in a data-driven fashion. Following GBST, the soft subword sequence is passed through Transformer layers.

출처: Charformer: Fast Character Transformers via Gradient-based Subword Tokenization

소개 논문: Charformer: Fast Character Transformers via Gradient-based Subword Tokenization