Actor-Critic Sequence Training for Image Captioning

2017-06-29 · Li Zhang, Flood Sung, Feng Liu, Tao Xiang, Shaogang Gong, Yongxin Yang, Timothy M. Hospedales

Generating natural language descriptions of images is an important capability for a robot or other visual-intelligence driven AI agent that may need to communicate with human users about what it is seeing. Such image captioning methods are typically trained by maximising the likelihood of ground-truth annotated caption given the image. While simple and easy to implement, this approach does not directly maximise the language quality metrics we care about such as CIDEr. In this paper we investigate training image captioning methods based on actor-critic reinforcement learning in order to directly optimise non-differentiable quality metrics of interest. By formulating a per-token advantage and value computation strategy in this novel reinforcement learning based captioning model, we show that it is possible to achieve the state of the art performance on the widely used MSCOCO benchmark.

📄 PDF Abstract BibTeX arXiv:1706.09601

Code (0)

등록된 구현이 없습니다.

Tasks

AI AgentImage Captioning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

B-SCST: Bayesian Self-Critical Sequence Training for Image Captioning

2020-04-06 · Shashank Bujimalla, Mahesh Subedar, Omesh Tickoo

Bayesian deep neural networks (DNNs) can provide a mathematically grounded framework to quantify uncertainty in predictions from image captioning models. We propose a Bayesian variant of policy-gradient based reinforceme…

Bayesian Inference Image Captioning Reinforcement Learning Uncertainty Quantification+1

Context-Aware Visual Policy Network for Sequence-Level Image Captioning

2018-08-16 · Daqing Liu, Zheng-Jun Zha, Hanwang Zhang, Yongdong Zhang 외

Many vision-language tasks can be reduced to the problem of sequence prediction for natural language output. In particular, recent advances in image captioning use deep reinforcement learning (RL) to alleviate the "expos…

Deep Reinforcement Learning Image Captioning Reinforcement Learning Reinforcement Learning (RL)

Self-critical n-step Training for Image Captioning

2019-04-15 · CVPR 2019 6 · Junlong Gao, Shiqi Wang, Shanshe Wang, Siwei Ma 외

Existing methods for image captioning are usually trained by cross entropy loss, which leads to exposure bias and the inconsistency between the optimizing function and evaluation metrics. Recently it has been shown that …

Image Captioning Reinforcement Learning

More Grounded Image Captioning by Distilling Image-Text Matching Model

2020-04-01 · CVPR 2020 6 · Yuanen Zhou, Meng Wang, Daqing Liu, Zhenzhen Hu 외

Visual attention not only improves the performance of image captioners, but also serves as a visual interpretation to qualitatively measure the caption rationality and model transparency. Specifically, we expect that a c…

Image Captioning Image-text matching Knowledge Distillation POS+2

Self-critical Sequence Training for Image Captioning

2016-12-02 · CVPR 2017 7 · Steven J. Rennie, Etienne Marcheret, Youssef Mroueh, Jarret Ross 외

Recently it has been shown that policy-gradient methods for reinforcement learning can be utilized to train deep end-to-end systems directly on non-differentiable metrics for the task at hand. In this paper we consider t…

Image Captioning Policy Gradient Methods reinforcement-learning Reinforcement Learning+1