A Cross-lingual Comparison of Human and Model Relative Word Importance
Relative word importance is a key metric for natural language processing. In this work, we compare human and model relative word importance to investigate if pretrained neural language models focus on the same words as humans cross-lingually. We perform an extensive study using several importance metrics (gradient-based saliency and attention-based) in monolingual and multilingual models, including eye-tracking corpora from four languages (German, Dutch, English, and Russian). We find that gradient-based saliency, first-layer attention, and attention flow correlate strongly with human eye-tracking data across all four languages. We further analyze the role of word length and word frequency in determining relative importance and find that it strongly correlates with length and frequency, however, the mechanisms behind these non-linear relations remain elusive. We obtain a cross-lingual approximation of the similarity between human and computational language processing and insights into the usability of several importance metrics.
Code (1)
Similar Papers 제목 키워드 기반
Graph Exploration and Cross-lingual Word Embeddings for Translation Inference Across Dictionaries
This paper describes the participation of two different approaches in the 3rd Translation Inference Across Dictionaries (TIAD 2020) shared task. The aim of the task is to automatically generate new bilingual dictionaries…
Cross-Lingual Word EmbeddingsTranslationWord EmbeddingsInvestigating Math Word Problems using Pretrained Multilingual Language Models
In this paper, we revisit math word problems~(MWPs) from the cross-lingual and multilingual perspective. We construct our MWP solvers over pretrained multilingual language models using sequence-to-sequence model with cop…
Machine TranslationMathPretrained Multilingual Language ModelsTranslationInvestigating Math Word Problems using Pretrained Multilingual Language Models
In this paper, we revisit math word problems~(MWPs) from the {\em cross-lingual} and {\em multilingual} perspective.We construct our MWP solvers over pretrained multilingual language models using the sequence-to-sequence…
Machine TranslationMathPretrained Multilingual Language ModelsTranslationAutomatic Learning of Modality Exclusivity Norms with Crosslingual Word Embeddings
Collecting modality exclusivity norms for lexical items has recently become a common practice in psycholinguistics and cognitive research. However, these norms are available only for a relatively small number of language…
Word EmbeddingsEvaluating Monolingual and Crosslingual Embeddings on Datasets of Word Association Norms
In free word association tasks, human subjects are presented with a stimulus word and are then asked to name the first word (the response word) that comes up to their mind. Those associations, presumably learned on the b…
Word Embeddings