Linguistic Query-Guided Mask Generation for Referring Image Segmentation
Referring image segmentation aims to segment the image region of interest according to the given language expression, which is a typical multi-modal task. Existing methods either adopt the pixel classification-based or the learnable query-based framework for mask generation, both of which are insufficient to deal with various text-image pairs with a fix number of parametric prototypes. In this work, we propose an end-to-end framework built on transformer to perform Linguistic query-Guided mask generation, dubbed LGFormer. It views the linguistic features as query to generate a specialized prototype for arbitrary input image-text pair, thus generating more consistent segmentation results. Moreover, we design several cross-modal interaction modules (\eg, vision-language bidirectional attention module, VLBA) in both encoder and decoder to achieve better cross-modal alignment.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive Learningcross-modal alignmentDecoderImage SegmentationSegmentationSemantic SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
RISAM: Referring Image Segmentation via Mutual-Aware Attention Features
Referring image segmentation (RIS) aims to segment a particular region based on a language expression prompt. Existing methods incorporate linguistic features into visual features and obtain multi-modal features for mask…
DecoderImage SegmentationSegmentationSemantic SegmentationLQMFormer: Language-aware Query Mask Transformer for Referring Image Segmentation
Referring Image Segmentation (RIS) aims to segment objects from an image based on a language description. Recent advancements have introduced transformer-based methods that leverage cross-modal dependencies significa…
Image SegmentationSemantic SegmentationVisual GroundingTemporal-Enhanced Multimodal Transformer for Referring Multi-Object Tracking and Segmentation
Referring multi-object tracking (RMOT) is an emerging cross-modal task that aims to locate an arbitrary number of target objects and maintain their identities referred by a language expression in a video. This intricate …
Multi-Object TrackingMulti-Object Tracking and SegmentationObject TrackingReferring Multi-Object Tracking+3RefMask3D: Language-Guided Transformer for 3D Referring Segmentation
3D referring segmentation is an emerging and challenging vision-language task that aims to segment the object described by a natural language expression in a point cloud scene. The key challenge behind this task is visio…
3D visual groundingImage SegmentationSegmentationSemantic Segmentation+1Contrastive Grouping with Transformer for Referring Image Segmentation
Referring image segmentation aims to segment the target referent in an image conditioning on a natural language expression. Existing one-stage methods employ per-pixel classification frameworks, which attempt straightfor…
Contrastive LearningImage SegmentationObjectSegmentation+1