Not All Neural Embeddings are Born Equal
Neural language models learn word representations that capture rich linguistic and conceptual information. Here we investigate the embeddings learned by neural machine translation models. We show that translation-based embeddings outperform those learned by cutting-edge monolingual models at single-language tasks requiring knowledge of conceptual similarity and/or syntactic role. The findings suggest that, while monolingual models learn information about how concepts are related, neural-translation models better capture their true ontological status.
Code (0)
등록된 구현이 없습니다.
Tasks
AllMachine TranslationTranslationSimilar Papers 제목 키워드 기반
Occupational Mobility: Theory and Estimation for Italy
This paper presents a model where intergenerational occupational mobility is the joint outcome of three main determinants: income incentives, equality of opportunity and changes in the composition of occupations. The mod…
Concentration in Gossip Opinion Dynamics over Random Graphs
We study concentration inequalities in gossip opinion dynamics over random graphs. In the model, a network is generated from a random graph model with independent edges, and agents interact pairwise randomly over the net…
Stochastic Block ModelNot all parameters are born equal: Attention is mostly what you need
Transformers are widely used in state-of-the-art machine translation, but the key to their success is still unknown. To gain insight into this, we consider three groups of parameters: embeddings, attention, and feed forw…
AllLanguage ModellingMachine TranslationTranslationOn modeling airborne infection risk
Airborne infection risk analysis is usually performed for enclosed spaces where susceptible individuals are exposed to infectious airborne respiratory droplets by inhalation. It is usually based on exponential, dose-resp…
From Flat Facts to Sharp Hallucinations: Detecting Stubborn Errors via Gradient Sensitivity
Traditional hallucination detection fails on "Stubborn Hallucinations" - errors where LLMs are confidently wrong. We propose a geometric solution: Embedding-Perturbed Gradient Sensitivity (EPGS). We hypothesize that whil…