Hierarchical Visual Primitive Experts for Compositional Zero-Shot Learning
Compositional zero-shot learning (CZSL) aims to recognize unseen compositions with prior knowledge of known primitives (attribute and object). Previous works for CZSL often suffer from grasping the contextuality between attribute and object, as well as the discriminability of visual features, and the long-tailed distribution of real-world compositional data. We propose a simple and scalable framework called Composition Transformer (CoT) to address these issues. CoT employs object and attribute experts in distinctive manners to generate representative embeddings, using the visual network hierarchically. The object expert extracts representative object embeddings from the final layer in a bottom-up manner, while the attribute expert makes attribute embeddings in a top-down manner with a proposed object-guided attention module that models contextuality explicitly. To remedy biased prediction caused by imbalanced data distribution, we develop a simple minority attribute augmentation (MAA) that synthesizes virtual samples by mixing two images and oversampling minority attribute classes. Our method achieves SoTA performance on several benchmarks, including MIT-States, C-GQA, and VAW-CZSL. We also demonstrate the effectiveness of CoT in improving visual discrimination and addressing the model bias from the imbalanced data distribution. The code is available at https://github.com/HanjaeKim98/CoT.
Code (1)
Tasks
AttributeCompositional Zero-Shot LearningObjectZero-Shot LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
EVA: Mixture-of-Experts Semantic Variant Alignment for Compositional Zero-Shot Learning
Compositional Zero-Shot Learning (CZSL) investigates compositional generalization capacity to recognize unknown state-object pairs based on learned primitive concepts. Existing CZSL methods typically derive primitives fe…
Compositional Zero-Shot LearningMixture-of-ExpertsZero-Shot LearningZero-shot Compositional Action Recognition with Neural Logic Constraints
Zero-shot compositional action recognition (ZS-CAR) aims to identify unseen verb-object compositions in the videos by exploiting the learned knowledge of verb and object primitives during training. Despite compositional …
Action RecognitionLearning Attention Propagation for Compositional Zero-Shot Learning
Compositional zero-shot learning aims to recognize unseen compositions of seen visual primitives of object classes and their states. While all primitives (states and objects) are observable during training in some combin…
Compositional Zero-Shot LearningZero-Shot LearningPrompting Language-Informed Distribution for Compositional Zero-Shot Learning
Compositional zero-shot learning (CZSL) task aims to recognize unseen compositional visual concepts, e.g., sliced tomatoes, where the model is learned only from the seen compositions, e.g., sliced potatoes and red tomato…
Compositional Zero-Shot LearningInformativenessZero-shot GeneralizationZero-Shot LearningLearning Graph Embeddings for Compositional Zero-shot Learning
In compositional zero-shot learning, the goal is to recognize unseen compositions (e.g. old dog) of observed visual primitives states (e.g. old, cute) and objects (e.g. car, dog) in the training set. This is challenging …
Compositional Zero-Shot LearningGraph EmbeddingTransfer LearningZero-Shot Learning