Large-Scale Evaluation of Keyphrase Extraction Models
Keyphrase extraction models are usually evaluated under different, not directly comparable, experimental setups. As a result, it remains unclear how well proposed models actually perform, and how they compare to each other. In this work, we address this issue by presenting a systematic large-scale analysis of state-of-the-art keyphrase extraction models involving multiple benchmark datasets from various sources and domains. Our main results reveal that state-of-the-art models are in fact still challenged by simple baselines on some datasets. We also present new insights about the impact of using author- or reader-assigned keyphrases as a proxy for gold standard, and give recommendations for strong baselines and reliable benchmark datasets.
Code (1)
Tasks
Keyphrase ExtractionSimilar Papers 제목 키워드 기반
Open Domain Web Keyphrase Extraction Beyond Language Modeling
This paper studies keyphrase extraction in real-world scenarios where documents are from diverse domains and have variant content quality. We curate and release OpenKP, a large scale open domain keyphrase extraction data…
Keyphrase ExtractionLanguage ModelingLanguage ModellingSlovKE: A Large-Scale Dataset and LLM Evaluation for Slovak Keyphrase Extraction
Keyphrase extraction for morphologically rich, low-resource languages remains understudied, largely due to the scarcity of suitable evaluation datasets. We address this gap for Slovak by constructing a dataset of 227,432…
Keyphrase ExtractionTermITH-Eval: a French Standard-Based Resource for Keyphrase Extraction Evaluation
Keyphrase extraction is the task of finding phrases that represent the important content of a document. The main aim of keyphrase extraction is to propose textual units that represent the most important topics developed …
Keyphrase ExtractionELSKE: Efficient Large-Scale Keyphrase Extraction
Keyphrase extraction methods can provide insights into large collections of documents such as social media posts. Existing methods, however, are less suited for the real-time analysis of streaming data, because they are …
Information RetrievalKeyphrase ExtractionKeyword ExtractionMulti-Document Summarization+1Theme-driven Keyphrase Extraction to Analyze Social Media Discourse
Social media platforms are vital resources for sharing self-reported health experiences, offering rich data on various health topics. Despite advancements in Natural Language Processing (NLP) enabling large-scale social …
Keyphrase ExtractionLanguage ModellingLarge Language Model