Sentence Encoding with Tree-constrained Relation Networks
The meaning of a sentence is a function of the relations that hold between its words. We instantiate this relational view of semantics in a series of neural models based on variants of relation networks (RNs) which represent a set of objects (for us, words forming a sentence) in terms of representations of pairs of objects. We propose two extensions to the basic RN model for natural language. First, building on the intuition that not all word pairs are equally informative about the meaning of a sentence, we use constraints based on both supervised and unsupervised dependency syntax to control which relations influence the representation. Second, since higher-order relations are poorly captured by a sum of pairwise relations, we use a recurrent extension of RNs to propagate information so as to form representations of higher order relations. Experiments on sentence classification, sentence pair classification, and machine translation reveal that, while basic RNs are only modestly effective for sentence representation, recurrent RNs with latent syntax are a reliably powerful representational device.
Code (0)
등록된 구현이 없습니다.
Tasks
General ClassificationMachine TranslationRelationSentenceSentence ClassificationSentence-Pair ClassificationTranslationSimilar Papers 제목 키워드 기반
Graph-tree Fusion Model with Bidirectional Information Propagation for Long Document Classification
Long document classification presents challenges in capturing both local and global dependencies due to their extensive content and complex structure. Existing methods often struggle with token limits and fail to adequat…
Document ClassificationGraph AttentionSentenceDependency Position Encoding for Relation Extraction
Leveraging the dependency tree of the input sentence is able to improve the model performance for relation extraction. A challenging issue is how to remove confusions from the tree. Efforts have been made to utilize the …
PositionRelationRelation ExtractionSentenceA Unifying Theory of Transition-based and Sequence Labeling Parsing
We define a mapping from transition-based parsing algorithms that read sentences from left to right to sequence labeling encodings of syntactic trees. This not only establishes a theoretical relation between transition-b…
Dependency ParsingSyntax Encoding with Application in Authorship Attribution
We propose a novel strategy to encode the syntax parse tree of sentence into a learnable distributed representation. The proposed syntax encoding scheme is provably information-lossless. In specific, an embedding vector …
Authorship AttributionBenchmarkingFeature EngineeringSentence+1Natural Language Inference by Tree-Based Convolution and Heuristic Matching
In this paper, we propose the TBCNN-pair model to recognize entailment and contradiction between two sentences. In our model, a tree-based convolutional neural network (TBCNN) captures sentence-level semantics; then heur…
Natural Language InferenceSentence