Stage-wise Fine-tuning for Graph-to-Text Generation
Graph-to-text generation has benefited from pre-trained language models (PLMs) in achieving better performance than structured graph encoders. However, they fail to fully utilize the structure information of the input graph. In this paper, we aim to further improve the performance of the pre-trained language model by proposing a structured graph-to-text model with a two-step fine-tuning mechanism which first fine-tunes the model on Wikipedia before adapting to the graph-to-text generation. In addition to using the traditional token and position embeddings to encode the knowledge graph (KG), we propose a novel tree-level embedding method to capture the inter-dependency structures of the input graph. This new approach has significantly improved the performance of all text generation metrics for the English WebNLG 2017 dataset.
Code (1)
Tasks
Data-to-Text GenerationKB-to-Language GenerationLanguage ModelingLanguage ModellingPositionText GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
GraphWalker: Agentic Knowledge Graph Question Answering via Synthetic Trajectory Curriculum
Agentic knowledge graph question answering (KGQA) requires an agent to iteratively interact with knowledge graphs (KGs), posing challenges in both training data scarcity and reasoning generalization. Specifically, existi…
Graph Question AnsweringReinforcement LearningKnowledge GraphsRankPO: Preference Optimization for Job-Talent Matching
Matching job descriptions (JDs) with suitable talent requires models capable of understanding not only textual similarities between JDs and candidate resumes but also contextual factors such as geographical location and …
Contrastive LearningExploring the Effectiveness of Multi-stage Fine-tuning for Cross-encoder Re-rankers
State-of-the-art cross-encoders can be fine-tuned to be highly effective in passage re-ranking. The typical fine-tuning process of cross-encoders as re-rankers requires large amounts of manually labelled data, a contrast…
Contrastive LearningLanguage ModelingLanguage ModellingLarge Language Model+2BLCU-NLP at COIN-Shared Task1: Stagewise Fine-tuning BERT for Commonsense Inference in Everyday Narrations
This paper describes our system for COIN Shared Task 1: Commonsense Inference in Everyday Narrations. To inject more external knowledge to better reason over the narrative passage, question and answer, the system adopts …
Machine Reading ComprehensionReading ComprehensionFine-tuning Multi-hop Question Answering with Hierarchical Graph Network
In this paper, we present a two stage model for multi-hop question answering. The first stage is a hierarchical graph network, which is used to reason over multi-hop question and is capable to capture different levels of…
Graph Neural NetworkLanguage ModelingLanguage ModellingMulti-hop Question Answering+1