paper-with-me

Papers

Distilling Event Sequence Knowledge From Large Language Models

2024-01-14 · Somin Wadhwa, Oktie Hassanzadeh, Debarun Bhattacharjya, Ken Barker, Jian Ni

Event sequence models have been found to be highly effective in the analysis and prediction of events. Building such models requires availability of abundant high-quality event sequence data. In certain applications, however, clean structured event sequences are not available, and automated sequence extraction results in data that is too noisy and incomplete. In this work, we explore the use of Large Language Models (LLMs) to generate event sequences that can effectively be used for probabilistic event model construction. This can be viewed as a mechanism of distilling event sequence knowledge from LLMs. Our approach relies on a Knowledge Graph (KG) of event concepts with partial causal relations to guide the generative language model for causal event sequence generation. We show that our approach can generate high-quality event sequences, filling a knowledge gap in the input KG. Furthermore, we explore how the generated sequences can be leveraged to discover useful and more complex structured knowledge from pattern mining and probabilistic event models. We release our sequence generation code and evaluation framework, as well as corpus of event sequence data.

📄 PDF Abstract BibTeX arXiv:2401.07237

Code (0)

등록된 구현이 없습니다.

Tasks

Language ModelingLanguage Modelling

Similar Papers 제목 키워드 기반

f-Divergence Minimization for Sequence-Level Knowledge Distillation

2023-07-27 · Yuqiao Wen, Zichao Li, Wenyu Du, Lili Mou

Knowledge distillation (KD) is the process of transferring knowledge from a large model to a small one. It has gained increasing attention in the natural language processing community, driven by the demands of compressin…

Knowledge Distillation

Distilling Discrimination and Generalization Knowledge for Event Detection via Delta-Representation Learning

2019-07-01 · ACL 2019 7 · Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun

Event detection systems rely on discrimination knowledge to distinguish ambiguous trigger words and generalization knowledge to detect unseen/sparse trigger words. Current neural event detection approaches focus on trigg…

Event DetectionRepresentation Learning

Distilling Multi-Scale Knowledge for Event Temporal Relation Extraction

2022-09-01 · Hao-Ren Yao, Luke Breitfeller, Aakanksha Naik, Chunxiao Zhou 외

Event Temporal Relation Extraction (ETRE) is paramount but challenging. Within a discourse, event pairs are situated at different distances or the so-called proximity bands. The temporal ordering communicated about event…

Contrastive LearningLanguage ModellingNatural Language UnderstandingRelation+2

Distilling Knowledge Learned in BERT for Text Generation

2019-11-10 · ACL 2020 6 · Yen-Chun Chen, Zhe Gan, Yu Cheng, Jingzhou Liu 외

Large-scale pre-trained language model such as BERT has achieved great success in language understanding tasks. However, it remains an open question how to utilize BERT for language generation. In this paper, we present …

Language ModelingLanguage ModellingMachine TranslationMasked Language Modeling+4

Hierarchical Knowledge Distillation for Dialogue Sequence Labeling

2021-11-22 · Shota Orihashi, Yoshihiro Yamazaki, Naoki Makishima, Mana Ihori 외

This paper presents a novel knowledge distillation method for dialogue sequence labeling. Dialogue sequence labeling is a supervised learning task that estimates labels for each utterance in the target dialogue document,…

Knowledge DistillationScene Segmentation