Simple, Fast Noise-Contrastive Estimation for Large RNN Vocabularies
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingMachine TranslationSpeech RecognitionSpelling CorrectionSimilar Papers 제목 키워드 기반
Learning word embeddings efficiently with noise-contrastive estimation
Continuous-valued word embeddings learned by neural language models have recently been shown to capture semantic and syntactic information about words very well, setting performance records on several word similarity tas…
Learning Word EmbeddingsWord EmbeddingsWord SimilarityImproved training of neural trans-dimensional random field language models with dynamic noise-contrastive estimation
A new whole-sentence language model - neural trans-dimensional random field language model (neural TRF LM), where sentences are modeled as a collection of random fields, and the potential function is defined by a neural …
Language ModelingLanguage ModellingSentenceLearning with Noise-Contrastive Estimation: Easing training by learning to scale
Noise-Contrastive Estimation (NCE) is a learning criterion that is regularly used to train neural language models in place of Maximum Likelihood Estimation, since it avoids the computational bottleneck caused by the outp…
Language ModelingLanguage ModellingMachine TranslationSpeech RecognitionBoltzNCE: Learning Likelihoods for Boltzmann Generation with Stochastic Interpolants and Noise Contrastive Estimation
Efficient sampling from the Boltzmann distribution given its energy function is a key challenge for modeling complex physical systems such as molecules. Boltzmann Generators address this problem by leveraging continuous …
Transfer LearningDo More Negative Samples Necessarily Hurt in Contrastive Learning?
Recent investigations in noise contrastive estimation suggest, both empirically as well as theoretically, that while having more "negative samples" in the contrastive loss improves downstream classification performance i…
Contrastive Learning