Zero-Shot Visual Recognition using Semantics-Preserving Adversarial Embedding Networks

2017-12-05 · CVPR 2018 6 · Long Chen, Hanwang Zhang, Jun Xiao, Wei Liu, Shih-Fu Chang

We propose a novel framework called Semantics-Preserving Adversarial Embedding Network (SP-AEN) for zero-shot visual recognition (ZSL), where test images and their classes are both unseen during training. SP-AEN aims to tackle the inherent problem --- semantic loss --- in the prevailing family of embedding-based ZSL, where some semantics would be discarded during training if they are non-discriminative for training classes, but could become critical for recognizing test classes. Specifically, SP-AEN prevents the semantic loss by introducing an independent visual-to-semantic space embedder which disentangles the semantic space into two subspaces for the two arguably conflicting objectives: classification and reconstruction. Through adversarial learning of the two subspaces, SP-AEN can transfer the semantics from the reconstructive subspace to the discriminative one, accomplishing the improved zero-shot recognition of unseen classes. Comparing with prior works, SP-AEN can not only improve classification but also generate photo-realistic images, demonstrating the effectiveness of semantic preservation. On four popular benchmarks: CUB, AWA, SUN and aPY, SP-AEN considerably outperforms other state-of-the-art methods by an absolute performance difference of 12.2\%, 9.3\%, 4.0\%, and 3.6\% in terms of harmonic mean values

📄 PDF Abstract BibTeX arXiv:1712.01928

Code (1)

MARMOTatZJU/ZSLPR-TIANCHI pytorch

Tasks

General Classification Zero-Shot Learning

Novel Semantic Prompting for Zero-Shot Action Recognition

2026-03-09 · Salman Iqbal, Waheed Rehman arxiv

Zero-shot action recognition relies on transferring knowledge from vision-language models to unseen actions using semantic descriptions. While recent methods focus on temporal modeling or architectural adaptations to han…

Zero-Shot Action Recognition Action Understanding

Zero-Shot Visual Recognition via Bidirectional Latent Embedding

2016-07-07 · Qian Wang, Ke Chen

Zero-shot learning for visual recognition, e.g., object and action recognition, has recently attracted a lot of attention. However, it still remains challenging in bridging the semantic gap between visual features and th…

Action Recognition Temporal Action Localization Zero-Shot Learning

Learning Using Privileged Information for Zero-Shot Action Recognition

2022-06-17 · Zhiyi Gao, Yonghong Hou, Wanqing Li, Zihui Guo 외

Zero-Shot Action Recognition (ZSAR) aims to recognize video actions that have never been seen during training. Most existing methods assume a shared semantic space between seen and unseen actions and intend to directly l…

Action Recognition Hallucination Object Zero-Shot Action Recognition

Dual Adversarial Semantics-Consistent Network for Generalized Zero-Shot Learning

2019-07-12 · NeurIPS 2019 12 · Jian Ni, Shanghang Zhang, Haiyong Xie

Generalized zero-shot learning (GZSL) is a challenging class of vision and knowledge transfer problems in which both seen and unseen classes appear during testing. Existing GZSL approaches either suffer from semantic los…

Generalized Zero-Shot Learning Transfer Learning Zero-Shot Learning

Low-Rank Embedded Ensemble Semantic Dictionary for Zero-Shot Learning

2017-07-01 · CVPR 2017 7 · Zhengming Ding, Ming Shao, Yun Fu

Zero-shot learning for visual recognition has received much interest in the most recent years. However, the semantic gap across visual features and their underlying semantics is still the biggest obstacle in zero-shot le…

Dictionary Learning Zero-Shot Learning