VolDoGer: LLM-assisted Datasets for Domain Generalization in Vision-Language Tasks
Domain generalizability is a crucial aspect of a deep learning model since it determines the capability of the model to perform well on data from unseen domains. However, research on the domain generalizability of deep learning models for vision-language tasks remains limited, primarily because of the lack of required datasets. To address these challenges, we propose VolDoGer: Vision-Language Dataset for Domain Generalization, a dedicated dataset designed for domain generalization that addresses three vision-language tasks: image captioning, visual question answering, and visual entailment. We constructed VolDoGer by extending LLM-based data annotation techniques to vision-language tasks, thereby alleviating the burden of recruiting human annotators. We evaluated the domain generalizability of various models, ranging from fine-tuned models to a recent multimodal large language model, through VolDoGer.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep LearningDomain GeneralizationImage CaptioningLanguage ModelingLanguage ModellingLarge Language ModelMultimodal Large Language ModelQuestion AnsweringVisual EntailmentVisual Question AnsweringSimilar Papers 제목 키워드 기반
WEDGE: Web-Image Assisted Domain Generalization for Semantic Segmentation
Domain generalization for semantic segmentation is highly demanded in real applications, where a trained model is expected to work well in previously unseen domains. One challenge lies in the lack of data which could cov…
DiversityDomain GeneralizationSegmentationSemantic SegmentationCo-Generation and Segmentation for Generalized Surgical Instrument Segmentation on Unlabelled Data
Surgical instrument segmentation for robot-assisted surgery is needed for accurate instrument tracking and augmented reality overlays. Therefore, the topic has been the subject of a number of recent papers in the CAI com…
SegmentationTranslationResonant Anomaly Detection with Multiple Reference Datasets
An important class of techniques for resonant anomaly detection in high energy physics builds models that can distinguish between reference and target datasets, where only the latter has appreciable signal. Such techniqu…
Anomaly DetectionGeneralized Radiograph Representation Learning via Cross-supervision between Images and Free-text Radiology Reports
Pre-training lays the foundation for recent successes in radiograph analysis supported by deep learning. It learns transferable image representations by conducting large-scale fully-supervised or self-supervised learning…
Representation LearningSelf-Supervised LearningTransfer LearningLMT-GP: Combined Latent Mean-Teacher and Gaussian Process for Semi-supervised Low-light Image Enhancement
While recent low-light image enhancement (LLIE) methods have made significant advancements, they still face challenges in terms of low visual quality and weak generalization ability when applied to complex scenarios. To …
GPRImage EnhancementLow-Light Image EnhancementPseudo Label