Probing sentence embeddings for structure-dependent tense
Learning universal sentence representations which accurately model sentential semantic content is a current goal of natural language processing research. A prominent and successful approach is to train recurrent neural networks (RNNs) to encode sentences into fixed length vectors. Many core linguistic phenomena that one would like to model in universal sentence representations depend on syntactic structure. Despite the fact that RNNs do not have explicit syntactic structural representations, there is some evidence that RNNs can approximate such structure-dependent phenomena under certain conditions, in addition to their widespread success in practical tasks. In this work, we assess RNNs{'} ability to learn the structure-dependent phenomenon of main clause tense.
Code (0)
등록된 구현이 없습니다.
Tasks
SentenceSentence EmbeddingsSimilar Papers 제목 키워드 기반
Evaluation of sentence embeddings in downstream and linguistic probing tasks
Despite the fast developmental pace of new sentence embedding methods, it is still challenging to find comprehensive evaluations of these different techniques. In the past years, we saw significant improvements in the fi…
Language ModelingLanguage ModellingSentenceSentence Embedding+3Vec2Sent: Probing Sentence Embeddings with Natural Language Generation
We introspect black-box sentence embeddings by conditionally generating from them with the objective to retrieve the underlying discrete sentence. We perceive of this as a new unsupervised probing task and show that it c…
SentenceSentence EmbeddingsText GenerationHow to Probe Sentence Embeddings in Low-Resource Languages: On Structural Design Choices for Probing Task Evaluation
Sentence encoders map sentences to real valued vectors for use in downstream applications. To peek into these representations - e.g., to increase interpretability of their results - probing tasks have been designed which…
SentenceSentence EmbeddingsLINSPECTOR WEB: A Multilingual Probing Suite for Word Representations
We present LINSPECTOR WEB, an open source multilingual inspector to analyze word representations. Our system provides researchers working in low-resource settings with an easily accessible web based probing tool to gain …
Dependency Parsingnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+2Empirical Linguistic Study of Sentence Embeddings
The purpose of the research is to answer the question whether linguistic information is retained in vector representations of sentences. We introduce a method of analysing the content of sentence embeddings based on univ…
SentenceSentence Embeddings