Revisiting Document Representations for Large-Scale Zero-Shot Learning
Zero-shot learning aims to recognize unseen objects using their semantic representations. Most existing works use visual attributes labeled by humans, not suitable for large-scale applications. In this paper, we revisit the use of documents as semantic representations. We argue that documents like Wikipedia pages contain rich visual information, which however can easily be buried by the vast amount of non-visual sentences. To address this issue, we propose a semi-automatic mechanism for visual sentence extraction that leverages the document section headers and the clustering structure of visual sentences. The extracted visual sentences, after a novel weighting scheme to distinguish similar classes, essentially form semantic representations like visual attributes but need much less human effort. On the ImageNet dataset with over 10,000 unseen classes, our representations lead to a 64% relative improvement against the commonly used ones.
Code (1)
Tasks
ClusteringSentenceZero-Shot LearningSimilar Papers 제목 키워드 기반
Zero-Shot Prompting and Few-Shot Fine-Tuning: Revisiting Document Image Classification Using Large Language Models
Classifying scanned documents is a challenging problem that involves image, layout, and text analysis for document understanding. Nevertheless, for certain benchmark datasets, notably RVL-CDIP, the state of the art is cl…
Document Classificationdocument-image-classificationDocument Image Classificationdocument understanding+2Revisiting the Centroid-based Method: A Strong Baseline for Multi-Document Summarization
The centroid-based model for extractive document summarization is a simple and fast baseline that ranks sentences based on their similarity to a centroid vector. In this paper, we apply this ranking to possible summaries…
Document SummarizationExtractive Document SummarizationExtractive Text SummarizationMulti-Document SummarizationScaleDoc: Scaling LLM-based Predicates over Large Document Collections
Predicates are foundational components in data analysis systems. However, modern workloads increasingly involve unstructured documents, which demands semantic understanding, beyond traditional value-based predicates. Giv…
Revisiting CLIP: Efficient Alignment of 3D MRI and Tabular Data using Domain-Specific Foundation Models
Multi-modal models require aligned, shared embedding spaces. However, common CLIP-based approaches need large amounts of samples and do not natively support 3D or tabular data, both of which are crucial in the medical do…
Image RetrievalRetrievalzero-shot-classificationZero-shot Image Retrieval+1Revisiting the Centroid-based Method: A Strong Baseline for Multi-Document Summarization
The centroid-based model for extractive document summarization is a simple and fast baseline that ranks sentences based on their similarity to a centroid vector. In this paper, we apply this ranking to possible summaries…
Document SummarizationExtractive Document SummarizationExtractive Text SummarizationMulti-Document Summarization