paper-with-me

홈 › Papers

StruBERT: Structure-aware BERT for Table Search and Matching

2022-03-27 · Mohamed Trabelsi, Zhiyu Chen, Shuo Zhang, Brian D. Davison, Jeff Heflin

A large amount of information is stored in data tables. Users can search for data tables using a keyword-based query. A table is composed primarily of data values that are organized in rows and columns providing implicit structural information. A table is usually accompanied by secondary information such as the caption, page title, etc., that form the textual information. Understanding the connection between the textual and structural information is an important yet neglected aspect in table retrieval as previous methods treat each source of information independently. In addition, users can search for data tables that are similar to an existing table, and this setting can be seen as a content-based table retrieval. In this paper, we propose StruBERT, a structure-aware BERT model that fuses the textual and structural information of a data table to produce context-aware representations for both textual and tabular content of a data table. StruBERT features are integrated in a new end-to-end neural ranking model to solve three table-related downstream tasks: keyword- and content-based table retrieval, and table similarity. We evaluate our approach using three datasets, and we demonstrate substantial improvements in terms of retrieval and classification metrics over state-of-the-art methods.

📄 PDF Abstract BibTeX arXiv:2203.14278

Code (1)

medtray/StruBERT 공식 구현 pytorch

Tasks

RetrievalTable RetrievalTable Search

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음
Attention 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
StruBERT A large amount of information is stored in data tables. Users can search for data tables using a keyword-based query. A table is composed primarily of data values that are…
Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…
Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…
Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…
Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Similar Papers 제목 키워드 기반

Table Search Using a Deep Contextualized Language Model

2020-05-19 · Zhiyu Chen, Mohamed Trabelsi, Jeff Heflin, Yinan Xu 외

Pretrained contextualized language models such as BERT have achieved impressive results on various natural language processing benchmarks. Benefiting from multiple pretraining tasks and large scale training corpora, pret…

Language ModelingLanguage ModellingRetrievalTable Retrieval+1

Context-Aware Legal Citation Recommendation using Deep Learning

2021-06-20 · Zihan Huang, Charles Low, Mengqiu Teng, Hongyi Zhang 외

Lawyers and judges spend a large amount of time researching the proper legal authority to cite while drafting decisions. In this paper, we develop a citation recommendation tool that can help improve efficiency in the pr…

Citation RecommendationCollaborative FilteringDeep Learningtext similarity

TabuLM: Morphology-Aware Tabular Pre-training for Low-Resource Languages

2026-08-27 · Ireddi Rakshitha, Devavarapu Yashwanth, Ntakirutimana Pierre arxiv

We present TabuLM, the first language model pre-trained on Kinyarwanda tabular data. Kinyarwanda is a morphologically rich Bantu language spoken by over 12 million people in Rwanda, yet lacks any dedicated tabular repres…

Representation LearningType prediction

SKILL: Similarity-aware Knowledge distILLation for Speech Self-Supervised Learning

2024-02-26 · Luca Zampierin, Ghouthi Boukli Hacene, Bac Nguyen, Mirco Ravanelli

Self-supervised learning (SSL) has achieved remarkable success across various speech-processing tasks. To enhance its efficiency, previous works often leverage the use of compression techniques. A notable recent attempt …

Knowledge DistillationSelf-Supervised Learning

HyperBERT: Mixing Hypergraph-Aware Layers with Language Models for Node Classification on Text-Attributed Hypergraphs

2024-02-11 · Adrián Bazaga, Pietro Liò, Gos Micklem

Hypergraphs are characterized by complex topological structure, representing higher-order interactions among multiple entities through hyperedges. Lately, hypergraph-based deep learning methods to learn informative data …

Inductive BiasLanguage ModelingLanguage ModellingNode Classification