Analyzing Bayesian Crosslingual Transfer in Topic Models
We introduce a theoretical analysis of crosslingual transfer in probabilistic topic models. By formulating posterior inference through Gibbs sampling as a process of language transfer, we propose a new measure that quantifies the loss of knowledge across languages during this process. This measure enables us to derive a PAC-Bayesian bound that elucidates the factors affecting model quality, both during training and in downstream applications. We provide experimental validation of the analysis on a diverse set of five languages, and discuss best practices for data collection and model design based on our analysis.
Code (0)
등록된 구현이 없습니다.
Tasks
Topic ModelsSimilar Papers 제목 키워드 기반
An Empirical Study on Crosslingual Transfer in Probabilistic Topic Models
Probabilistic topic modeling is a popular choice as the first step of crosslingual tasks to enable knowledge transfer and extract multilingual features. While many multilingual topic models have been developed, their ass…
Topic ModelsTransfer LearningCrosslingual Topic Modeling with WikiPDA
We present Wikipedia-based Polyglot Dirichlet Allocation (WikiPDA), a crosslingual topic model that learns to represent Wikipedia articles written in any language as distributions over a common set of language-independen…
ArticlesMatrix CompletionA Closer Look at Few-Shot Crosslingual Transfer: The Choice of Shots Matters
Few-shot crosslingual transfer has been shown to outperform its zero-shot counterpart with pretrained encoders like multilingual BERT. Despite its growing popularity, little to no attention has been paid to standardizing…
Few-Shot LearningLearning Multilingual Topics from Incomparable Corpus
Multilingual topic models enable crosslingual tasks by extracting consistent topics from multilingual corpora. Most models require parallel or comparable training corpora, which limits their ability to generalize. In thi…
Topic ModelsTransfer LearningLearning Multilingual Topics from Incomparable Corpora
Multilingual topic models enable crosslingual tasks by extracting consistent topics from multilingual corpora. Most models require parallel or comparable training corpora, which limits their ability to generalize. In thi…
Topic ModelsTransfer Learning