Table Search Using a Deep Contextualized Language Model
Pretrained contextualized language models such as BERT have achieved impressive results on various natural language processing benchmarks. Benefiting from multiple pretraining tasks and large scale training corpora, pretrained models can capture complex syntactic word relations. In this paper, we use the deep contextualized language model BERT for the task of ad hoc table retrieval. We investigate how to encode table content considering the table structure and input length limit of BERT. We also propose an approach that incorporates features from prior literature on table retrieval and jointly trains them with BERT. In experiments on public datasets, we show that our best approach can outperform the previous state-of-the-art method and BERT baselines with a large margin under different evaluation metrics.
Code (1)
Tasks
Language ModelingLanguage ModellingRetrievalTable RetrievalTable SearchMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Negation, Coordination, and Quantifiers in Contextualized Language Models
With the success of contextualized language models, much research explores what these models really learn and in which cases they still fail. Most of this work focuses on specific NLP tasks and on the learning outcome. L…
NegationBERT, mBERT, or BiBERT? A Study on Contextualized Embeddings for Neural Machine Translation
The success of bidirectional encoders using masked language models, such as BERT, on numerous natural language processing tasks has prompted researchers to attempt to incorporate these pre-trained models into neural mach…
de-enLanguage ModelingLanguage ModellingMachine Translation+2Self-prompting and cross-model consensus enable reproducible data extraction from scientific literature with large language models
Accurately extracting nuanced, contextualized data from research articles is laborious and time intensive. Here, we investigate the performance of frontier, browser-based large language models (LLMs) to extract highly co…
What Does This Word Mean? Explaining Contextualized Embeddings with Natural Language Definition
Contextualized word embeddings have boosted many NLP tasks compared with traditional static word embeddings. However, the word with a specific sense may have different contextualized embeddings due to its various context…
Word EmbeddingsConQueR: Contextualized Query Reduction using Search Logs
Query reformulation is a key mechanism to alleviate the linguistic chasm of query in ad-hoc retrieval. Among various solutions, query reduction effectively removes extraneous terms and specifies concise user intent from …
Language ModelingLanguage ModellingRetrievalTerm Extraction