Evolving Semantic Prototype Improves Generative Zero-Shot Learning

2023-06-12 · Shiming Chen, Wenjin Hou, Ziming Hong, Xiaohan Ding, Yibing Song, Xinge You, Tongliang Liu, Kun Zhang

In zero-shot learning (ZSL), generative methods synthesize class-related sample features based on predefined semantic prototypes. They advance the ZSL performance by synthesizing unseen class sample features for better training the classifier. We observe that each class's predefined semantic prototype (also referred to as semantic embedding or condition) does not accurately match its real semantic prototype. So the synthesized visual sample features do not faithfully represent the real sample features, limiting the classifier training and existing ZSL performance. In this paper, we formulate this mismatch phenomenon as the visual-semantic domain shift problem. We propose a dynamic semantic prototype evolving (DSP) method to align the empirically predefined semantic prototypes and the real prototypes for class-related feature synthesis. The alignment is learned by refining sample features and semantic prototypes in a unified framework and making the synthesized visual sample features approach real sample features. After alignment, synthesized sample features from unseen classes are closer to the real sample features and benefit DSP to improve existing generative ZSL methods by 8.5\%, 8.0\%, and 9.7\% on the standard CUB, SUN AWA2 datasets, the significant performance improvement indicates that evolving semantic prototype explores a virgin field in ZSL.

📄 PDF Abstract BibTeX arXiv:2306.06931

Code (0)

등록된 구현이 없습니다.

Tasks

Zero-Shot Learning

Methods 이 논문이 사용한 방법론

ALIGN In the ALIGN method, visual and language representations are jointly trained from noisy image alt-text data. The image and text encoders are learned via contrastive loss…

Visual-Augmented Dynamic Semantic Prototype for Generative Zero-Shot Learning

2024-04-23 · CVPR 2024 1 · Wenjin Hou, Shiming Chen, Shuhuang Chen, Ziming Hong 외

Generative Zero-shot learning (ZSL) learns a generator to synthesize visual samples for unseen classes, which is an effective way to advance ZSL. However, existing generative methods rely on the conditions of Gaussian no…

Zero-Shot Learning

Neuron: Learning Context-Aware Evolving Representations for Zero-Shot Skeleton Action Recognition

2024-11-18 · CVPR 2025 1 · Yang Chen, Jingcai Guo, Song Guo, DaCheng Tao

Zero-shot skeleton action recognition is a non-trivial task that requires robust unseen generalization with prior knowledge from only seen classes and shared semantics. Existing methods typically build the skeleton-seman…

Action Recognition

Zero-Shot Learning with Generative Latent Prototype Model

2017-05-26 · Yanan Li, Donghui Wang

Zero-shot learning, which studies the problem of object classification for categories for which we have no training examples, is gaining increasing attention from community. Most existing ZSL methods exploit deterministi…

model Transfer Learning Zero-Shot Learning

Incentivizing Generative Zero-Shot Learning via Outcome-Reward Reinforcement Learning with Visual Cues

2026-03-22 · Wenjin Hou, Xiaoxiao Sun, Hehe Fan arxiv

Recent advances in zero-shot learning (ZSL) have demonstrated the potential of generative models. Typically, generative ZSL synthesizes visual features conditioned on semantic prototypes to model the data distribution of…

Reinforcement Learning Zero-Shot Learning

Episode-based Prototype Generating Network for Zero-Shot Learning

2019-09-08 · CVPR 2020 6 · Yunlong Yu, Zhong Ji, Zhongfei Zhang, Jungong Han

We introduce a simple yet effective episode-based training framework for zero-shot learning (ZSL), where the learning system requires to recognize unseen classes given only the corresponding class semantics. During train…

zero-shot-classification Zero-Shot Learning