The Solution for the 5th GCAIAC Zero-shot Referring Expression Comprehension Challenge
This report presents a solution for the zero-shot referring expression comprehension task. Visual-language multimodal base models (such as CLIP, SAM) have gained significant attention in recent years as a cornerstone of mainstream research. One of the key applications of multimodal base models lies in their ability to generalize to zero-shot downstream tasks. Unlike traditional referring expression comprehension, zero-shot referring expression comprehension aims to apply pre-trained visual-language models directly to the task without specific training. Recent studies have enhanced the zero-shot performance of multimodal base models in referring expression comprehension tasks by introducing visual prompts. To address the zero-shot referring expression comprehension challenge, we introduced a combination of visual prompts and considered the influence of textual prompts, employing joint prediction tailored to the data characteristics. Ultimately, our approach achieved accuracy rates of 84.825 on the A leaderboard and 71.460 on the B leaderboard, securing the first position.
Code (0)
등록된 구현이 없습니다.
Tasks
Referring ExpressionReferring Expression ComprehensionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Zero-shot Referring Image Segmentation with Global-Local Context Features
Referring image segmentation (RIS) aims to find a segmentation mask given a referring expression grounded to a region of the input image. Collecting labelled datasets for this task, however, is notoriously costly and lab…
Image SegmentationReferring ExpressionReferring Expression SegmentationSegmentation+3Text Augmented Spatial-aware Zero-shot Referring Image Segmentation
In this paper, we study a challenging task of zero-shot referring image segmentation. This task aims to identify the instance mask that is most related to a referring expression without training on pixel-level annotation…
Image SegmentationReferring ExpressionSegmentationSemantic Segmentation+2ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehension
Training a referring expression comprehension (ReC) model for a new visual domain requires collecting referring expressions, and potentially corresponding bounding boxes, for images in the domain. While large-scale pre-t…
image-classificationImage ClassificationReferring ExpressionReferring Expression Comprehension+1ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehension
Training a referring expression comprehension (ReC) model for a new visual domain requires collecting referring expressions, and potentially corresponding bounding boxes, for images in the domain. While large-scale pre-t…
image-classificationImage ClassificationReferring ExpressionReferring Expression Comprehension+1Zero-Shot Referring Expression Comprehension via Vison-Language True/False Verification
Referring Expression Comprehension (REC) is usually addressed with task-trained grounding models. We show that a zero-shot workflow, without any REC-specific training, can achieve competitive or superior performance. Our…
Referring Expression