Sentence Embeddings for Russian NLU

2019-10-29 · Dmitry Popov, Alexander Pugachev, Polina Svyatokum, Elizaveta Svitanko, Ekaterina Artemova

We investigate the performance of sentence embeddings models on several tasks for the Russian language. In our comparison, we include such tasks as multiple choice question answering, next sentence prediction, and paraphrase identification. We employ FastText embeddings as a baseline and compare it to ELMo and BERT embeddings. We conduct two series of experiments, using both unsupervised (i.e., based on similarity measure only) and supervised approaches for the tasks. Finally, we present datasets for multiple choice question answering and next sentence prediction in Russian.

📄 PDF Abstract BibTeX arXiv:1910.13291

Code (1)

avidale/encodechka

Tasks

Multiple-choiceParaphrase Identification Prediction Question AnsweringSentenceSentence Embeddings

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Sigmoid Activation 설명 없음

Tanh Activation 설명 없음

Residual Connection 설명 없음

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Weight Decay 설명 없음

Refunds@Expedia|||How do I get a full refund from Expedia? “How do I get a full refund from Expedia? How do I get a full refund from Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Quick Help &…

LANGUAGE MODEL EMBEDDINGS IMPROVE SENTIMENT ANALYSIS IN RUSSIAN

2019-05-29 · Computational Linguistics and Intellectual Technologies: Proceedings of the International Conference “Dialogue 2019” 2019 5 · Baymurzina D. R., Kuznetsov D. P., Burtsev M. S.

Sentiment analysis is one of the most popular natural language processing tasks. In this paper we introduce pre-trained Russian language models which are used to extract embeddings (ELMo) to improve accuracy for classifi…

Articles Language Modeling Language Modelling model+2

Transformers for Headline Selection for Russian News Clusters

2021-06-19 · Pavel Voropaev, Olga Sopilnyak

In this paper, we explore various multilingual and Russian pre-trained transformer-based models for the Dialogue Evaluation 2021 shared task on headline selection. Our experiments show that the combined approach is super…

Dialogue Evaluation Sentence Sentence Embeddings

Can Eye Movement Data Be Used As Ground Truth For Word Embeddings Evaluation?

2018-04-23 · Amir Bakarov

In recent years a certain success in the task of modeling lexical semantics was obtained with distributional semantic models. Nevertheless, the scientific community is still unaware what is the most reliable evaluation m…

Embeddings Evaluation Sentence Word Embeddings Word Similarity

GPT-3.5 for Grammatical Error Correction

2024-05-14 · Anisia Katinskaia, Roman Yangarber

This paper investigates the application of GPT-3.5 for Grammatical Error Correction (GEC) in multiple languages in several settings: zero-shot GEC, fine-tuning for GEC, and using GPT-3.5 to re-rank correction hypotheses …

Grammatical Error Correction Sentence

Automated WordNet Construction Using Word Embeddings

2017-04-01 · WS 2017 4 · Mikhail Khodak, Andrej Risteski, Christiane Fellbaum, Sanjeev Arora

We present a fully unsupervised method for automated construction of WordNets based upon recent advances in distributional representations of sentences and word-senses combined with readily available machine translation …

Information Retrieval Machine Translation Translation Word Embeddings+1