Modeling Explicit Concerning States for Reinforcement Learning in Visual Dialogue

2021-07-12 · Zipeng Xu, Fandong Meng, Xiaojie Wang, Duo Zheng, Chenxu Lv, Jie zhou

To encourage AI agents to conduct meaningful Visual Dialogue (VD), the use of Reinforcement Learning has been proven potential. In Reinforcement Learning, it is crucial to represent states and assign rewards based on the action-caused transitions of states. However, the state representation in previous Visual Dialogue works uses the textual information only and its transitions are implicit. In this paper, we propose Explicit Concerning States (ECS) to represent what visual contents are concerned at each round and what have been concerned throughout the Visual Dialogue. ECS is modeled from multimodal information and is represented explicitly. Based on ECS, we formulate two intuitive and interpretable rewards to encourage the Visual Dialogue agents to converse on diverse and informative visual information. Experimental results on the VisDial v1.0 dataset show our method enables the Visual Dialogue agents to generate more visual coherent, less repetitive and more visual informative dialogues compared with previous methods, according to multiple automatic metrics, human study and qualitative analysis.

📄 PDF Abstract BibTeX arXiv:2107.05250

Code (1)

zipengxuc/ecs-visdial-rl 공식 구현 pytorch

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

WildWorld: A Large-Scale Dataset for Dynamic World Modeling with Actions and Explicit State toward Generative ARPG

2026-03-24 · Zhen Li, Zian Meng, Shuwei Shi, Wenshuo Peng 외 arxiv

Dynamical systems theory and reinforcement learning view world evolution as latent-state dynamics driven by actions, with visual observations providing partial information about the state. Recent video world models attem…

Reinforcement Learning Video Generation

MOOSS: Mask-Enhanced Temporal Contrastive Learning for Smooth State Evolution in Visual Reinforcement Learning

2024-09-02 · Jiarui Sun, M. Ugur Akcal, Wei zhang, Girish Chowdhary

In visual Reinforcement Learning (RL), learning from pixel-based observations poses significant challenges on sample efficiency, primarily due to the complexity of extracting informative state representations from high-d…

Contrastive Learning graph construction Reinforcement Learning (RL)

Cascaded LSTMs based Deep Reinforcement Learning for Goal-driven Dialogue

2019-10-31 · Yue Ma, Xiaojie Wang, Zhenjiang Dong, Hong Chen

This paper proposes a deep neural network model for joint modeling Natural Language Understanding (NLU) and Dialogue Management (DM) in goal-driven dialogue systems. There are three parts in this model. A Long Short-Term…

Deep Reinforcement Learning Dialogue Management Management Natural Language Understanding+3

Controllable Flow Matching for Online Reinforcement Learning

2025-11-10 · Bin Wang, Boxiang Tao, Haifeng Jing, Hongbo Dou 외 arxiv

Model-based reinforcement learning (MBRL) typically relies on modeling environment dynamics for data efficiency. However, due to the accumulation of model errors over long-horizon rollouts, such methods often face challe…

Reinforcement Learning

Obstacle Tower Without Human Demonstrations: How Far a Deep Feed-Forward Network Goes with Reinforcement Learning

2020-04-01 · Marco Pleines, Jenia Jitsev, Mike Preuss, Frank Zimmer

The Obstacle Tower Challenge is the task to master a procedurally generated chain of levels that subsequently get harder to complete. Whereas the most top performing entries of last year's competition used human demonstr…

Deep Reinforcement Learning Reinforcement Learning