A Simple but Effective Pluggable Entity Lookup Table for Pre-trained Language Models
Pre-trained language models (PLMs) cannot well recall rich factual knowledge of entities exhibited in large-scale corpora, especially those rare entities. In this paper, we propose to build a simple but effective Pluggable Entity Lookup Table (PELT) on demand by aggregating the entity's output representations of multiple occurrences in the corpora. PELT can be compatibly plugged as inputs to infuse supplemental entity knowledge into PLMs. Compared to previous knowledge-enhanced PLMs, PELT only requires 0.2%-5% pre-computation with capability of acquiring knowledge from out-of-domain corpora for domain adaptation scenario. The experiments on knowledge-related tasks demonstrate that our method, PELT, can flexibly and effectively transfer entity knowledge from related corpora into PLMs with different architectures.
Code (1)
Tasks
Domain AdaptationSimilar Papers 제목 키워드 기반
A Simple but Effective Pluggable Entity Lookup Table for Pre-trained Language Models
Pre-trained language models (PLMs) cannot well recall rich factual knowledge of entities exhibited in large-scale corpora, especially those rare entities. In this paper, we propose to build a simple but effective Pluggab…
Domain AdaptationMatching Web Tables with Knowledge Base Entities: From Entity Lookups to Entity Embeddings
Web tables constitute valuable sources of information for various applications, ranging from Web search to Knowledge Base (KB) augmentation. An underlying common requirement is to annotate the rows of Web tables with sem…
Cell Entity AnnotationEntity EmbeddingsOntology MatchingTable annotationFaster Convolution Inference Through Using Pre-Calculated Lookup Tables
Low-cardinality activations permit an algorithm based on fetching the inference values from pre-calculated lookup tables instead of calculating them every time. This algorithm can have extensions, some of which offer abi…
A Pluggable Common Sense-Enhanced Framework for Knowledge Graph Completion
Knowledge graph completion (KGC) tasks aim to infer missing facts in a knowledge graph (KG) for many knowledge-intensive applications. However, existing embedding-based KGC approaches primarily rely on factual triples, p…
Common Sense ReasoningGraph EmbeddingKnowledge Graph CompletionKnowledge Graph EmbeddingOn-the-fly Table Generation
Many information needs revolve around entities, which would be better answered by summarizing results in a tabular format, rather than presenting them as a ranked list. Unlike previous work, which is limited to retrievin…