WiCo: Win-win Cooperation of Bottom-up and Top-down Referring Image Segmentation
The top-down and bottom-up methods are two mainstreams of referring segmentation, while both methods have their own intrinsic weaknesses. Top-down methods are chiefly disturbed by Polar Negative (PN) errors owing to the lack of fine-grained cross-modal alignment. Bottom-up methods are mainly perturbed by Inferior Positive (IP) errors due to the lack of prior object information. Nevertheless, we discover that two types of methods are highly complementary for restraining respective weaknesses but the direct average combination leads to harmful interference. In this context, we build Win-win Cooperation (WiCo) to exploit complementary nature of two types of methods on both interaction and integration aspects for achieving a win-win improvement. For the interaction aspect, Complementary Feature Interaction (CFI) provides fine-grained information to top-down branch and introduces prior object information to bottom-up branch for complementary feature enhancement. For the integration aspect, Gaussian Scoring Integration (GSI) models the gaussian performance distributions of two branches and weightedly integrates results by sampling confident scores from the distributions. With our WiCo, several prominent top-down and bottom-up combinations achieve remarkable improvements on three common datasets with reasonable extra costs, which justifies effectiveness and generality of our method.
Code (0)
등록된 구현이 없습니다.
Tasks
cross-modal alignmentImage SegmentationReferring Expression SegmentationSemantic SegmentationSimilar Papers 제목 키워드 기반
Unpaired Referring Expression Grounding via Bidirectional Cross-Modal Matching
Referring expression grounding is an important and challenging task in computer vision. To avoid the laborious annotation in conventional referring grounding, unpaired referring grounding is introduced, where the trainin…
Image-text matchingReferring ExpressionText MatchingLanguage Controls More Than Top-Down Attention: Modulating Bottom-Up Visual Processing with Referring Expressions
How to best integrate linguistic and perceptual processing in multimodal tasks is an important open problem. In this work we argue that the common technique of using language to direct visual attention over high-level vi…
Referring ExpressionRethinking Cross-modal Interaction from a Top-down Perspective for Referring Video Object Segmentation
Referring video object segmentation (RVOS) aims to segment video objects with the guidance of natural language reference. Previous methods typically tackle RVOS through directly grounding linguistic reference over the im…
ObjectOne-shot visual object segmentationReferring Video Object SegmentationSemantic Segmentation+2Modulating Bottom-Up and Top-Down Visual Processing via Language-Conditional Filters
How to best integrate linguistic and perceptual processing in multi-modal tasks that involve language and vision is an important open problem. In this work, we argue that the common practice of using language in a top-do…
ColorizationImage ColorizationReferring ExpressionReferring Expression Segmentation+1See-Through-Text Grouping for Referring Image Segmentation
Motivated by the conventional grouping techniques to image segmentation, we develop their DNN counterpart to tackle the referring variant. The proposed method is driven by a convolutional-recurrent neural network (ConvRN…
Image Segmentationobject-detectionObject DetectionReferring Expression+4