EntropyRank: Unsupervised Keyphrase Extraction via Side-Information Optimization for Language Model-based Text Compression
We propose an unsupervised method to extract keywords and keyphrases from texts based on a pre-trained language model (LM) and Shannon's information maximization. Specifically, our method extracts phrases having the highest conditional entropy under the LM. The resulting set of keyphrases turns out to solve a relevant information-theoretic problem: if provided as side information, it leads to the expected minimal binary code length in compressing the text using the LM and an entropy encoder. Alternately, the resulting set is an approximation via a causal LM to the set of phrases that minimize the entropy of the text when conditioned upon it. Empirically, the method provides results comparable to the most commonly used methods in various keyphrase extraction benchmark challenges.
Code (0)
등록된 구현이 없습니다.
Tasks
Keyphrase ExtractionLanguage ModelingLanguage ModellingText CompressionSimilar Papers 제목 키워드 기반
Unsupervised Keyphrase Extraction via Interpretable Neural Networks
Keyphrase extraction aims at automatically extracting a list of "important'' phrases which represent the key concepts in a document. Traditionally, it has been approached from an information-theoretic angle using phrase …
ArticlesKeyphrase ExtractionLocal Word Vectors Guiding Keyphrase Extraction
Automated keyphrase extraction is a fundamental textual information processing task concerned with the selection of representative phrases from a document that summarize its content. This work presents a novel unsupervis…
Keyphrase ExtractionWord EmbeddingsPatternRank: Leveraging Pretrained Language Models and Part of Speech for Unsupervised Keyphrase Extraction
Keyphrase extraction is the process of automatically selecting a small set of most relevant phrases from a given text. Supervised keyphrase extraction approaches need large amounts of labeled training data and perform po…
Keyphrase ExtractionUnsupervised Keyphrase Extraction with Multipartite Graphs
We propose an unsupervised keyphrase extraction model that encodes topical information within a multipartite graph structure. Our model represents keyphrase candidates and topics in a single graph and exploits their mutu…
Keyphrase ExtractionWikiRank: Improving Keyphrase Extraction Based on Background Knowledge
Keyphrase is an efficient representation of the main idea of documents. While background knowledge can provide valuable information about documents, they are rarely incorporated in keyphrase extraction methods. In this p…
Keyphrase Extraction