Self-Supervised Pre-Training for Table Structure Recognition Transformer
Table structure recognition (TSR) aims to convert tabular images into a machine-readable format. Although hybrid convolutional neural network (CNN)-transformer architecture is widely used in existing approaches, linear projection transformer has outperformed the hybrid architecture in numerous vision tasks due to its simplicity and efficiency. However, existing research has demonstrated that a direct replacement of CNN backbone with linear projection leads to a marked performance drop. In this work, we resolve the issue by proposing a self-supervised pre-training (SSP) method for TSR transformers. We discover that the performance gap between the linear projection transformer and the hybrid CNN-transformer can be mitigated by SSP of the visual encoder in the TSR model. We conducted reproducible ablation studies and open-sourced our code at https://github.com/poloclub/unitable to enhance transparency, inspire innovations, and facilitate fair comparisons in our domain as tables are a promising modality for representation learning.
Code (1)
Tasks
Representation LearningSimilar Papers 제목 키워드 기반
UniTable: Towards a Unified Framework for Table Recognition via Self-Supervised Pretraining
Tables convey factual and quantitative data with implicit conventions created by humans that are often challenging for machines to parse. Prior work on table recognition (TR) has mainly centered around complex task-speci…
Language ModelingLanguage ModellingTable RecognitionA Study on Self-Supervised Pretraining for Vision Problems in Gastrointestinal Endoscopy
Solutions to vision tasks in gastrointestinal endoscopy (GIE) conventionally use image encoders pretrained in a supervised manner with ImageNet-1k as backbones. However, the use of modern self-supervised pretraining algo…
Depth EstimationLandmark RecognitionMonocular Depth EstimationProsodic Structure Beyond Lexical Content: A Study of Self-Supervised Learning
People exploit the predictability of lexical structures during text comprehension. Though predictable structure is also present in speech, the degree to which prosody, e.g. intonation, tempo, and loudness, contributes to…
Emotion RecognitionReading ComprehensionSelf-Supervised LearningSelf-supervised Learning for ECG-based Emotion Recognition
We present an electrocardiogram (ECG) -based emotion recognition system using self-supervised learning. Our proposed architecture consists of two main networks, a signal transformation recognition network and an emotion …
Emotion RecognitionSelf-Supervised LearningRegCLR: A Self-Supervised Framework for Tabular Representation Learning in the Wild
Recent advances in self-supervised learning (SSL) using large models to learn visual representations from natural images are rapidly closing the gap between the results produced by fully supervised learning and those pro…
Representation LearningSelf-Supervised LearningTable Recognition