SHYI: Action Support for Contrastive Learning in High-Fidelity Text-to-Image Generation

2025-01-15 · Tianxiang Xia, Lin Xiao, Yannick Montorfani, Francesco Pavia, Enis Simsar, Thomas Hofmann

In this project, we address the issue of infidelity in text-to-image generation, particularly for actions involving multiple objects. For this we build on top of the CONFORM framework which uses Contrastive Learning to improve the accuracy of the generated image for multiple objects. However the depiction of actions which involves multiple different object has still large room for improvement. To improve, we employ semantically hypergraphic contrastive adjacency learning, a comprehension of enhanced contrastive structure and "contrast but link" technique. We further amend Stable Diffusion's understanding of actions by InteractDiffusion. As evaluation metrics we use image-text similarity CLIP and TIFA. In addition, we conducted a user study. Our method shows promising results even with verbs that Stable Diffusion understands mediocrely. We then provide future directions by analyzing the results. Our codebase can be found on polybox under the link: https://polybox.ethz.ch/index.php/s/dJm3SWyRohUrFxn

📄 PDF Abstract BibTeX arXiv:2501.09055

Code (0)

등록된 구현이 없습니다.

Tasks

Contrastive Learning Image Generationtext similarityText to Image Generation Text-to-Image Generation

Methods 이 논문이 사용한 방법론

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Contrastive Learning 설명 없음

CLIP Contrastive Language-Image Pre-training (CLIP), consisting of a simplified version of ConVIRT trained from scratch, is an efficient method of image representation learning…

Tree Bandits for Generative Bayes

2024-04-16 · Sean O'Hagan, Jungeum Kim, Veronika Rockova

In generative models with obscured likelihood, Approximate Bayesian Computation (ABC) is often the tool of last resort for inference. However, ABC demands many prior parameter trials to keep only a small fraction that pa…

image-classification Image Classification

High-Fidelity One-Step Generative Visuomotor Policy via Recursive Correction, Frequency Consistency, and Contrastive Flow Matching

2026-07-04 · Yuran Chen, Xinye Cai, Zhonglin Gong, Yang Huang arxiv

Generative models such as diffusion and flow matching have advanced robotic visuomotor policies by modeling multimodal action distributions, but their multi-step sampling or ODE solving introduces inference latency. Exis…

High Fidelity Text to Image Generation with Contrastive Alignment and Structural Guidance

2025-08-14 · Danyi Gao arxiv

This paper addresses the performance bottlenecks of existing text-driven image generation methods in terms of semantic alignment accuracy and structural consistency. A high-fidelity image generation method is proposed by…

Contrastive Learning Image Generation

Covariance Structure and Coordinate Heterogeneity Govern Binary Quantization of Contrastive Embeddings

2026-05-17 · Wenxuan Xiao arxiv

Binary quantization (BQ) compresses high-dimensional embeddings into one or two bits per coordinate, enabling nearest neighbor search at extreme speed. Yet a striking puzzle persists: BQ achieves competitive recall on co…

Contrastive Context-Aware Learning for 3D High-Fidelity Mask Face Presentation Attack Detection

2021-04-13 · Ajian Liu, Chenxu Zhao, Zitong Yu, Jun Wan 외

Face presentation attack detection (PAD) is essential to secure face recognition systems primarily from high-fidelity mask attacks. Most existing 3D mask PAD benchmarks suffer from several drawbacks: 1) a limited number …

Face Presentation Attack Detection Face Recognition