Representation Learning for Resource-Constrained Keyphrase Generation
State-of-the-art keyphrase generation methods generally depend on large annotated datasets, limiting their performance in domains with limited annotated data. To overcome this challenge, we design a data-oriented approach that first identifies salient information using retrieval-based corpus-level statistics, and then learns a task-specific intermediate representation based on a pre-trained language model using large-scale unlabeled documents. We introduce salient span recovery and salient span prediction as denoising training objectives that condense the intra-article and inter-article knowledge essential for keyphrase generation. Through experiments on multiple keyphrase generation benchmarks, we show the effectiveness of the proposed approach for facilitating low-resource keyphrase generation and zero-shot domain adaptation. Our method especially benefits the generation of absent keyphrases, approaching the performance of models trained with large training sets.
Code (1)
Tasks
DenoisingDomain AdaptationKeyphrase GenerationLanguage ModelingLanguage ModellingRepresentation LearningRetrievalSimilar Papers 제목 키워드 기반
Representation Learning for Resource-Constrained Keyphrase Generation
State-of-the-art keyphrase generation methods generally depend on large annotated datasets, limiting their performance in domains with constrained resources. To overcome this challenge, we investigate pre-training strate…
DenoisingKeyphrase GenerationLanguage ModelingLanguage Modelling+1Data Augmentation for Low-Resource Keyphrase Generation
Keyphrase generation is the task of summarizing the contents of any given article into a few salient phrases (or keyphrases). Existing works for the task mostly rely on large-scale annotated datasets, which are not easy …
ArticlesData AugmentationKeyphrase GenerationKPDrop: Improving Absent Keyphrase Generation
Keyphrase generation is the task of generating phrases (keyphrases) that summarize the main topics of a given document. Keyphrases can be either present or absent from the given document. While the extraction of present …
Keyphrase GenerationAutoregressive Language Model for Zero-shot Constrained Keyphrase Generation
Recently, most of the state-of-the-art keyphrase prediction models are based on a supervised generative model. It shows significantly better than before. Nevertheless, it still faces domain robustness and building datase…
Keyphrase GenerationLanguage ModelingLanguage ModellingCausal Language Model for Zero-shot Constrained Keyphrase Generation
Recently, most of the state-of-the-art keyphrase prediction models are based on a supervised generative model.Although it shows noticeable improvement over statistical methods, it still struggles with low performance on …
Keyphrase GenerationLanguage ModelingLanguage Modelling