Image Captioning based on Deep Reinforcement Learning
Recently it has shown that the policy-gradient methods for reinforcement learning have been utilized to train deep end-to-end systems on natural language processing tasks. What's more, with the complexity of understanding image content and diverse ways of describing image content in natural language, image captioning has been a challenging problem to deal with. To the best of our knowledge, most state-of-the-art methods follow a pattern of sequential model, such as recurrent neural networks (RNN). However, in this paper, we propose a novel architecture for image captioning with deep reinforcement learning to optimize image captioning tasks. We utilize two networks called "policy network" and "value network" to collaboratively generate the captions of images. The experiments are conducted on Microsoft COCO dataset, and the experimental results have verified the effectiveness of the proposed method.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Reinforcement LearningImage CaptioningPolicy Gradient Methodsreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Actor-Critic Sequence Training for Image Captioning
Generating natural language descriptions of images is an important capability for a robot or other visual-intelligence driven AI agent that may need to communicate with human users about what it is seeing. Such image cap…
AI AgentImage Captioningreinforcement-learningReinforcement Learning+1Re$^3$Cap: Retrieval-Guided Refinement for Image Captioning Enhancement via Reinforcement Learning
Reinforcement Learning (RL) has demonstrated significant gains in image captioning, yet it is still limited in encouraging Large Vision-Language Models (LVLMs) to explore novel reasoning strategies. This limitation leads…
Reinforcement LearningImage CaptioningMulti-Level Policy and Reward Reinforcement Learning for Image Captioning
Image captioning is one of the most challenging hallmarks of AI, due to its complexity in visual and natural language understanding. As it is essentially a sequential prediction task, recent advances in image captioning …
Image CaptioningNatural Language Understandingreinforcement-learningReinforcement Learning+2Contextualized Keyword Representations for Multi-modal Retinal Image Captioning
Medical image captioning automatically generates a medical description to describe the content of a given medical image. A traditional medical image captioning model creates a medical description only based on a single m…
AvgImage CaptioningMulti-modal reward for visual relationships-based image captioning
Deep neural networks have achieved promising results in automatic image captioning due to their effective representation learning and context-based content generation capabilities. As a prominent type of deep features us…
Caption GenerationDeep Reinforcement LearningImage CaptioningModel Optimization+3