Semantically Consistent Regularization for Zero-Shot Recognition
The role of semantics in zero-shot learning is considered. The effectiveness of previous approaches is analyzed according to the form of supervision provided. While some learn semantics independently, others only supervise the semantic subspace explained by training classes. Thus, the former is able to constrain the whole space but lacks the ability to model semantic correlations. The latter addresses this issue but leaves part of the semantic space unsupervised. This complementarity is exploited in a new convolutional neural network (CNN) framework, which proposes the use of semantics as constraints for recognition.Although a CNN trained for classification has no transfer ability, this can be encouraged by learning an hidden semantic layer together with a semantic code for classification. Two forms of semantic constraints are then introduced. The first is a loss-based regularizer that introduces a generalization constraint on each semantic predictor. The second is a codeword regularizer that favors semantic-to-class mappings consistent with prior semantic knowledge while allowing these to be learned from data. Significant improvements over the state-of-the-art are achieved on several datasets.
Code (1)
Tasks
General ClassificationZero-Shot LearningSimilar Papers 제목 키워드 기반
Beyond Templates: Revisiting Zero-Shot Remote Sensing through Meta-Prompting
Vision-language models (VLMs) have sparked growing interest in zero-shot Earth Observation (EO) downstream tasks, with further gains enabled by remote-sensing-adapted models. We examine this setting across 17 VLM variant…
Learning Visually Consistent Label Embeddings for Zero-Shot Learning
In this work, we propose a zero-shot learning method to effectively model knowledge transfer between classes via jointly learning visually consistent word vectors and label embedding model in an end-to-end manner. The ma…
Transfer LearningZero-Shot LearningCross-modal Representation Learning for Zero-shot Action Recognition
We present a cross-modal Transformer-based framework, which jointly encodes video data and text labels for zero-shot action recognition (ZSAR). Our model employs a conceptually new pipeline by which visual representation…
Action RecognitionRepresentation LearningZero-Shot Action RecognitionHIT-SCIR at MMNLU-22: Consistency Regularization for Multilingual Spoken Language Understanding
Multilingual spoken language understanding (SLU) consists of two sub-tasks, namely intent detection and slot filling. To improve the performance of these two sub-tasks, we propose to use consistency regularization based …
Data AugmentationIntent Detectionslot-fillingSlot Filling+1Context-Aware Zero-Shot Recognition
We present a novel problem setting in zero-shot learning, zero-shot object recognition and detection in the context. Contrary to the traditional zero-shot learning methods, which simply infers unseen categories by transf…
Object RecognitionZero-Shot Learning