Weakly-supervised Compositional FeatureAggregation for Few-shot Recognition
Learning from a few examples is a challenging task for machine learning. While recent progress has been made for this problem, most of the existing methods ignore the compositionality in visual concept representation (e.g. objects are built from parts or composed of semantic attributes), which is key to the human ability to easily learn from a small number of examples. To enhance the few-shot learning models with compositionality, in this paper we present the simple yet powerful Compositional Feature Aggregation (CFA) module as a weakly-supervised regularization for deep networks. Given the deep feature maps extracted from the input, our CFA module first disentangles the feature space into disjoint semantic subspaces that model different attributes, and then bilinearly aggregates the local features within each of these subspaces. CFA explicitly regularizes the representation with both semantic and spatial compositionality to produce discriminative representations for few-shot recognition tasks. Moreover, our method does not need any supervision for attributes and object parts during training, thus can be conveniently plugged into existing models for end-to-end optimization while keeping the model size and computation cost nearly the same. Extensive experiments on few-shot image classification and action recognition tasks demonstrate that our method provides substantial improvements over recent state-of-the-art methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionFew-Shot Image ClassificationFew-Shot Learningimage-classificationImage ClassificationSimilar Papers 제목 키워드 기반
Revisiting Weakly Supervised Pre-Training of Visual Perception Models
Model pre-training is a cornerstone of modern visual recognition systems. Although fully supervised pre-training on datasets like ImageNet is still the de-facto standard, recent studies suggest that large-scale weakly su…
Fine-Grained Image ClassificationImage ClassificationOut-of-Distribution GeneralizationSelf-Supervised Learning+2STPro: Spatial and Temporal Progressive Learning for Weakly Supervised Spatio-Temporal Grounding
In this work, we study Weakly Supervised Spatio-Temporal Video Grounding (WSTVG), a challenging task of localizing subjects spatio-temporally in videos using only textual queries and no bounding box supervision. Insp…
Action UnderstandingSpatio-Temporal Video GroundingVideo GroundingCompositional Embeddings for Multi-Label One-Shot Learning
We present a compositional embedding framework that infers not just a single class per input image, but a set of classes, in the setting of one-shot learning. Specifically, we propose and evaluate several novel models co…
Object DetectionObject RecognitionOne-Shot LearningSpeaker DiarizationExploring the Spectrum of Visio-Linguistic Compositionality and Recognition
Vision and language models (VLMs) such as CLIP have showcased remarkable zero-shot recognition abilities yet face challenges in visio-linguistic compositionality, particularly in linguistic comprehension and fine-grained…
Retrievalzero-shot-classificationZero-Shot LearningCompositionally Equivariant Representation Learning
Deep learning models often need sufficient supervision (i.e. labelled data) in order to be trained effectively. By contrast, humans can swiftly learn to identify important anatomy in medical images like MRI and CT scans,…
AnatomyImage SegmentationMedical Image SegmentationRepresentation Learning+1