Modeling Graph Structure via Relative Position for Text Generation from Knowledge Graphs
We present Graformer, a novel Transformer-based encoder-decoder architecture for graph-to-text generation. With our novel graph self-attention, the encoding of a node relies on all nodes in the input graph - not only direct neighbors - facilitating the detection of global patterns. We represent the relation between two nodes as the length of the shortest path between them. Graformer learns to weight these node-node relations differently for different attention heads, thus virtually learning differently connected views of the input graph. We evaluate Graformer on two popular graph-to-text generation benchmarks, AGENDA and WebNLG, where it achieves strong performance while using many fewer parameters than other approaches.
Code (0)
등록된 구현이 없습니다.
Tasks
Data-to-Text GenerationDecoderKG-to-Text GenerationKnowledge GraphsPositionText GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
3D-RPE: Enhancing Long-Context Modeling Through 3D Rotary Position Encoding
Inspired by the Bloch Sphere representation, we propose a novel rotary position encoding on a three-dimensional sphere, named 3D Rotary Position Encoding (3D-RPE). 3D-RPE is an advanced version of the widely used 2D Rota…
Language ModelingLanguage ModellingNatural Language UnderstandingPositionInvestigating the Effect of Relative Positional Embeddings on AMR-to-Text Generation with Structural Adapters
Text generation from Abstract Meaning Representation (AMR) has substantially benefited from the popularized Pretrained Language Models (PLMs). Myriad approaches have linearized the input graph as a sequence of tokens to …
Abstract Meaning RepresentationAMR-to-Text GenerationLink PredictionPosition+1S+PAGE: A Speaker and Position-Aware Graph Neural Network Model for Emotion Recognition in Conversation
Emotion recognition in conversation (ERC) has attracted much attention in recent years for its necessity in widespread applications. Existing ERC methods mostly model the self and inter-speaker context separately, posing…
Emotion RecognitionEmotion Recognition in ConversationGraph Neural NetworkPositionUnifying Structured Data as Graph for Data-to-Text Pre-Training
Data-to-text (D2T) generation aims to transform structured data into natural language text. Data-to-text pre-training has proved to be powerful in enhancing D2T generation and yields impressive performances. However, pre…
Data-to-Text GenerationText GenerationMeT: A Graph Transformer for Semantic Segmentation of 3D Meshes
Polygonal meshes have become the standard for discretely approximating 3D shapes, thanks to their efficiency and high flexibility in capturing non-uniform shapes. This non-uniformity, however, leads to irregularity in th…
SegmentationSemantic Segmentation