paper-with-me

홈 › Papers

ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation

2023-04-12 · NeurIPS 2023 11 · Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, Yuxiao Dong

We present a comprehensive solution to learn and improve text-to-image models from human preference feedback. To begin with, we build ImageReward -- the first general-purpose text-to-image human preference reward model -- to effectively encode human preferences. Its training is based on our systematic annotation pipeline including rating and ranking, which collects 137k expert comparisons to date. In human evaluation, ImageReward outperforms existing scoring models and metrics, making it a promising automatic metric for evaluating text-to-image synthesis. On top of it, we propose Reward Feedback Learning (ReFL), a direct tuning algorithm to optimize diffusion models against a scorer. Both automatic and human evaluation support ReFL's advantages over compared methods. All code and datasets are provided at \url{https://github.com/THUDM/ImageReward}.

📄 PDF Abstract BibTeX arXiv:2304.05977

Code (3)

thudm/imagereward 공식 구현 pytorch
AMD-AIG-AIMA/ReNeg pytorch
LemonTwoL/ReNeg pytorch

Tasks

Image GenerationPreference MappingText to Image GenerationText-to-Image Generation

Methods 이 논문이 사용한 방법론

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Similar Papers 제목 키워드 기반

Test-Time Reasoning Through Visual Human Preferences with VLMs and Soft Rewards

2025-03-25 · Alexander Gambashidze, Konstantin Sobolev, Andrey Kuznetsov, Ivan Oseledets

Can Visual Language Models (VLMs) effectively capture human visual preferences? This work addresses this question by training VLMs to think about preferences at test time, employing reinforcement learning methods inspire…

World Knowledge

Listener-Rewarded Thinking in VLMs for Image Preferences

2025-06-28 · Alexander Gambashidze, Li Pengyi, Matvey Skripkin, Andrey Galichin 외

Training robust and generalizable reward models for human visual preferences is essential for aligning text-to-image and text-to-video generative models with human intent. However, current reward models often fail to gen…

MemorizationReinforcement Learning (RL)

Aligning Diffusion Models by Optimizing Human Utility

2024-04-06 · Shufan Li, Konstantinos Kallidromitis, Akash Gokul, Yusuke Kato 외

We present Diffusion-KTO, a novel approach for aligning text-to-image diffusion models by formulating the alignment objective as the maximization of expected human utility. Since this objective applies to each generation…

MIRO: MultI-Reward cOnditioned pretraining improves T2I quality and efficiency

2025-10-29 · Nicolas Dufour, Lucas Degeorge, Arijit Ghosh, Vicky Kalogeiton 외 arxiv

The default paradigm of post-training text-to-image generators includes post-hoc selection of generated images, and subsequent training with one reward model to align the generator to the reward, typically user preferenc…

Evaluating Text-to-Image Synthesis with a Conditional Fréchet Distance

2025-03-27 · Jaywon Koo, Jefferson Hernandez, Moayed Haji-Ali, Ziyan Yang 외

Evaluating text-to-image synthesis is challenging due to misalignment between established metrics and human preferences. We propose cFreD, a metric based on the notion of Conditional Fr\'echet Distance that explicitly ac…

BenchmarkingImage Generation