Multi-Level Policy and Reward Reinforcement Learning for Image Captioning

2018-06-15 · IJCAI 2018 6 · An-An Liu1, Ning Xu1, Hanwang Zhang2, Weizhi Nie1, Yuting Su1, Yongdong Zhang

Image captioning is one of the most challenging hallmarks of AI, due to its complexity in visual and natural language understanding. As it is essentially a sequential prediction task, recent advances in image captioning use Reinforcement Learning (RL) to better explore the dynamics of word-by-word generation. However, existing RL-based image captioning methods mainly rely on a single policy network and reward function that does not well fit the multi-level (word and sentence) and multi-modal (vision and language) nature of the task. To this end, we propose a novel multi-level policy and reward RL framework for image captioning. It contains two modules: 1) Multi-Level Policy Network that can adaptively fuse the word-level policy and the sentence-level policy for the word generation; and 2) Multi-Level Reward Function that collaboratively leverages both vision-language reward and language-language reward to guide the policy. Further, we propose a guidance term to bridge the policy and the reward for RL optimization. Extensive experiments and analysis on MSCOCO and Flick- r30k show that the proposed framework can achieve competing performances with respect to different evaluation metrics.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Image Captioning Natural Language Understanding reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Sentence

Learn Goal-Conditioned Policy with Intrinsic Motivation for Deep Reinforcement Learning

2021-04-11 · Jinxin Liu, Donglin Wang, Qiangxing Tian, Zhengyu Chen

It is of significance for an agent to learn a widely applicable and general-purpose policy that can achieve diverse goals including images and text descriptions. Considering such perceptually-specific goals, the frontier…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)

Skill-Critic: Refining Learned Skills for Hierarchical Reinforcement Learning

2023-06-14 · Ce Hao, Catherine Weaver, Chen Tang, Kenta Kawamoto 외

Hierarchical reinforcement learning (RL) can accelerate long-horizon decision-making by temporally abstracting a policy into multiple levels. Promising results in sparse reward environments have been seen with skills, i.…

Autonomous Racing Decision Making Hierarchical Reinforcement Learning reinforcement-learning+2

Hierarchical Reinforcement Learning with Advantage-Based Auxiliary Rewards

2019-10-10 · NeurIPS 2019 12 · Siyuan Li, Rui Wang, Minxue Tang, Chongjie Zhang

Hierarchical Reinforcement Learning (HRL) is a promising approach to solving long-horizon problems with sparse and delayed rewards. Many existing HRL algorithms either use pre-trained low-level skills that are unadaptabl…

Hierarchical Reinforcement Learning MuJoCo reinforcement-learning Reinforcement Learning+1

MIRG-RL: Multi-Image Reasoning and Grounding with Reinforcement Learning

2025-09-26 · Lihao Zheng, Jiawei Chen, Xintian Shen, Hao Ma 외 arxiv

Multi-image reasoning and grounding require understanding complex cross-image relationships at both object levels and image levels. Current Large Visual Language Models (LVLMs) face two critical challenges: the lack of c…

Reinforcement Learning

RbRL2.0: Integrated Reward and Policy Learning for Rating-based Reinforcement Learning

2025-01-13 · Mingkang Wu, Devin White, Vernon Lawhern, Nicholas R. Waytowich 외

Reinforcement learning (RL), a common tool in decision making, learns policies from various experiences based on the associated cumulative return/rewards without treating them differently. On the contrary, humans often l…

Decision Making Reinforcement Learning (RL)