A Simple Geometric Method for Cross-Lingual Linguistic Transformations with Pre-trained Autoencoders
Powerful sentence encoders trained for multiple languages are on the rise. These systems are capable of embedding a wide range of linguistic properties into vector representations. While explicit probing tasks can be used to verify the presence of specific linguistic properties, it is unclear whether the vector representations can be manipulated to indirectly steer such properties. For efficient learning, we investigate the use of a geometric mapping in embedding space to transform linguistic properties, without any tuning of the pre-trained sentence encoder or decoder. We validate our approach on three linguistic properties using a pre-trained multilingual autoencoder and analyze the results in both monolingual and cross-lingual settings.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderSentenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
When Language Representations Interact: Separability and Cross-Lingual Effects in LLMs
Large language models exhibit strong multilingual capabilities, however, their internal representations are difficult to interpret. Understanding these interactions is important for ensuring reliable behavior in multilin…
Mapping Semantic & Syntactic Relationships with Geometric Rotation
Understanding how language and embedding models encode semantic relationships is fundamental to model interpretability. While early word embeddings exhibited intuitive vector arithmetic (''king'' - ''man'' + ''woman'' = …
Language Model Metrics and Procrustes Analysis for Improved Vector Transformation of NLP Embeddings
Artificial Neural networks are mathematical models at their core. This truismpresents some fundamental difficulty when networks are tasked with Natural Language Processing. A key problem lies in measuring the similarity …
Language ModelingLanguage ModellingCross-neutralising: Probing for joint encoding of linguistic information in multilingual models
Multilingual sentence encoders are widely used to transfer NLP models across languages. The success of this transfer is, however, dependent on the model's ability to encode the patterns of cross-lingual similarity and va…
SentenceXLM-RPretrained Multilingual Transformers Reveal Quantitative Distance Between Human Languages
Understanding the distance between human languages is central to linguistics, anthropology, and tracing human evolutionary history. Yet, while linguistics has long provided rich qualitative accounts of cross-linguistic v…
Machine Translation