Learning Disentangled Representations for Natural Language Definitions
Disentangling the encodings of neural models is a fundamental aspect for improving interpretability, semantic control and downstream task performance in Natural Language Processing. Currently, most disentanglement methods are unsupervised or rely on synthetic datasets with known generative factors. We argue that recurrent syntactic and semantic regularities in textual data can be used to provide the models with both structural biases and generative factors. We leverage the semantic structures present in a representative and semantically dense category of sentence types, definitional sentences, for training a Variational Autoencoder to learn disentangled representations. Our experimental results show that the proposed model outperforms unsupervised baselines on several qualitative and quantitative benchmarks for disentanglement, and it also improves the results in the downstream task of definition modeling.
Code (0)
등록된 구현이 없습니다.
Tasks
DisentanglementSentenceSimilar Papers 제목 키워드 기반
Learning Disentangled Representations in Natural Language Definitions with Semantic Role Labeling Supervision
Disentangling the encodings of neural models is a fundamental aspect for improving interpretability, semantic control and downstream task performance in Natural Language Processing. However, most disentanglement methods …
DisentanglementSemantic Role LabelingSentenceDisentangled Representation Learning
Disentangled Representation Learning (DRL) aims to learn a model capable of identifying and disentangling the underlying factors hidden in the observable data in representation form. The process of separating underlying …
Representation LearningA Framework for the Quantitative Evaluation of Disentangled Representations
Recent AI research has emphasised the importance of learning disentangled representations of the explanatory factors behind data. Despite the growing interest in models which can learn such representations, visual inspe…
Enriching Disentanglement: From Logical Definitions to Quantitative Metrics
Disentangling the explanatory factors in complex data is a promising approach for generalizable and data-efficient representation learning. While a variety of quantitative metrics for learning and evaluating disentangled…
DisentanglementRepresentation LearningExplaining Word Embeddings via Disentangled Representation
Disentangled representations have attracted increasing attention recently. However, how to transfer the desired properties of disentanglement to word representations is unclear. In this work, we propose to transform typi…
DisentanglementWord Embeddings