Incorporate Dependency Relation Knowledge into Transformer Block for Multi-turn Dialogue Generation
Because of the compositionality of natural language, syntactic structure is one of the key factors for semantic understanding. However, the Transformer block, which is widely used for obtaining the distributed representations of sentences in dialogue generation tasks, views sentences as a sequence of words and does not effectively learn the syntactic structure. In this work, we explore how to effectively incorporate dependency relation knowledge that contains syntactic structure information into Transformer block and propose Dependency Relation Attention(DRA). Experimental results demonstrate that DRA can further improve the performance of state-of-the-art models for multi-turn dialogue generation.
Code (0)
등록된 구현이 없습니다.
Tasks
Dialogue GenerationRelationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Source Dependency-Aware Transformer with Supervised Self-Attention
Recently, Transformer has achieved the state-of-the-art performance on many machine translation tasks. However, without syntax knowledge explicitly considered in the encoder, incorrect context information that violates t…
Machine TranslationTranslationDependency-Based Self-Attention for Transformer NMT
In this paper, we propose a new Transformer neural machine translation (NMT) model that incorporates dependency relations into self-attention on both source and target sides, dependency-based self-attention. The dependen…
DecoderMachine TranslationNMTSemantic Role Labeling+1Dependency Position Encoding for Relation Extraction
Leveraging the dependency tree of the input sentence is able to improve the model performance for relation extraction. A challenging issue is how to remove confusions from the tree. Efforts have been made to utilize the …
PositionRelationRelation ExtractionSentenceTransformer with Syntactic Position Encoding for Machine Translation
It has been widely recognized that syntax information can help end-to-end neural machine translation (NMT) systems to achieve better translation. In order to integrate dependency information into Transformer based NMT, e…
Machine TranslationNMTPositionRelation+1DocTime: A Document-level Temporal Dependency Graph Parser
We introduce DocTime - a novel temporal dependency graph (TDG) parser that takes as input a text document and produces a temporal dependency graph. It outperforms previous BERT-based solutions by a relative 4-8% on three…