Contrastive Training Improves Zero-Shot Classification of Semi-structured Documents
We investigate semi-structured document classification in a zero-shot setting. Classification of semi-structured documents is more challenging than that of standard unstructured documents, as positional, layout, and style information play a vital role in interpreting such documents. The standard classification setting where categories are fixed during both training and testing falls short in dynamic environments where new document categories could potentially emerge. We focus exclusively on the zero-shot setting where inference is done on new unseen classes. To address this task, we propose a matching-based approach that relies on a pairwise contrastive objective for both pretraining and fine-tuning. Our results show a significant boost in Macro F$_1$ from the proposed pretraining step in both supervised and unsupervised zero-shot settings.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationDocument Classificationzero-shot-classificationZero-Shot LearningSimilar Papers 제목 키워드 기반
PESCO: Prompt-enhanced Self Contrastive Learning for Zero-shot Text Classification
We present PESCO, a novel contrastive learning framework that substantially improves the performance of zero-shot text classification. We formulate text classification as a neural text matching problem where each documen…
ClassificationContrastive Learningtext-classificationText Classification+2Significantly improving zero-shot X-ray pathology classification via fine-tuning pre-trained image-text encoders
Deep neural networks are increasingly used in medical imaging for tasks such as pathological classification, but they face challenges due to the scarcity of high-quality, expert-labeled training data. Recent efforts have…
ClassificationContrastive LearningSentencezero-shot-classification+1OFF-CLIP: Improving Normal Detection Confidence in Radiology CLIP with Simple Off-Diagonal Term Auto-Adjustment
Contrastive Language-Image Pre-Training (CLIP) has enabled zero-shot classification in radiology, reducing reliance on manual annotations. However, conventional contrastive learning struggles with normal case detection d…
Anomaly LocalizationClassificationClusteringContrastive Learning+3Modeling Caption Diversity in Contrastive Vision-Language Pretraining
There are a thousand ways to caption an image. Contrastive Language Pretraining (CLIP) on the other hand, works by mapping an image and its caption to a single vector -- limiting how well CLIP-like models can represent t…
Diversityzero-shot-classificationZero-Shot LearningCICA: Content-Injected Contrastive Alignment for Zero-Shot Document Image Classification
Zero-shot learning has been extensively investigated in the broader field of visual recognition, attracting significant interest recently. However, the current work on zero-shot learning in document image classification …
Document Classificationdocument-image-classificationDocument Image ClassificationGeneralized Zero-Shot Learning+3