RevCD -- Reversed Conditional Diffusion for Generalized Zero-Shot Learning
In Generalized Zero-Shot Learning (GZSL), we aim to recognize both seen and unseen categories using a model trained only on seen categories. In computer vision, this translates into a classification problem, where knowledge from seen categories is transferred to unseen categories by exploiting the relationships between visual features and available semantic information, such as text corpora or manual annotations. However, learning this joint distribution is costly and requires one-to-one training with corresponding semantic information. We present a reversed conditional Diffusion-based model (RevCD) that mitigates this issue by generating semantic features synthesized from visual inputs by leveraging Diffusion models' conditional mechanisms. Our RevCD model consists of a cross Hadamard-Addition embedding of a sinusoidal time schedule and a multi-headed visual transformer for attention-guided embeddings. The proposed approach introduces three key innovations. First, we reverse the process of generating semantic space based on visual data, introducing a novel loss function that facilitates more efficient knowledge transfer. Second, we apply Diffusion models to zero-shot learning - a novel approach that exploits their strengths in capturing data complexity. Third, we demonstrate our model's performance through a comprehensive cross-dataset evaluation. The complete code will be available on GitHub.
Code (0)
등록된 구현이 없습니다.
Tasks
Generalized Zero-Shot LearningTransfer LearningZero-Shot LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
DensePure: Understanding Diffusion Models towards Adversarial Robustness
Diffusion models have been recently employed to improve certified robustness through the process of denoising. However, the theoretical understanding of why diffusion models are able to improve the certified robustness i…
Adversarial RobustnessDenoisingSteered Diffusion: A Generalized Framework for Plug-and-Play Conditional Image Synthesis
Conditional generative models typically demand large annotated training sets to achieve high-quality synthesis. As a result, there has been significant interest in designing models that perform plug-and-play generation, …
ColorizationConditional Image GenerationImage GenerationImage Super-Resolution+2Stability of mixed-state phases under weak decoherence
We prove that the Gibbs states of classical, and commuting-Pauli, Hamiltonians are stable under weak local decoherence: i.e., we show that the effect of the decoherence can be locally reversed. In particular, our conclus…
Modiff: Action-Conditioned 3D Motion Generation with Denoising Diffusion Probabilistic Models
Diffusion-based generative models have recently emerged as powerful solutions for high-quality synthesis in multiple domains. Leveraging the bidirectional Markov chains, diffusion probabilistic models generate samples by…
DenoisingMotion GenerationAnalysis of Diffusion Models for Manifold Data
We analyze the time reversed dynamics of generative diffusion models. If the exact empirical score function is used in a regime of large dimension and exponentially large number of samples, these models are known to unde…