Learning to predict where to look in interactive environments using deep recurrent q-learning

2016-12-17 · Sajad Mousavi, Michael Schukat, Enda Howley, Ali Borji, Nasser Mozayani

Bottom-Up (BU) saliency models do not perform well in complex interactive environments where humans are actively engaged in tasks (e.g., sandwich making and playing the video games). In this paper, we leverage Reinforcement Learning (RL) to highlight task-relevant locations of input frames. We propose a soft attention mechanism combined with the Deep Q-Network (DQN) model to teach an RL agent how to play a game and where to look by focusing on the most pertinent parts of its visual input. Our evaluations on several Atari 2600 games show that the soft attention based model could predict fixation locations significantly better than bottom-up models such as Itti-Kochs saliency and Graph-Based Visual Saliency (GBVS) models.

📄 PDF Abstract BibTeX arXiv:1612.05753

Code (0)

등록된 구현이 없습니다.

Tasks

Atari GamesQ-Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

An Interactive Musical Prediction System with Mixture Density Recurrent Neural Networks

2019-04-10 · Charles P. Martin, Jim Torresen

This paper is about creating digital musical instruments where a predictive neural network model is integrated into the interactive system. Rather than predicting symbolic music (e.g., MIDI notes), we suggest that predic…

ProAct: Agentic Lookahead in Interactive Environments

2026-02-05 · Yangbin Yu, Mingyu Yang, Junyou Li, Yiming Gao 외 arxiv

Existing Large Language Model (LLM) agents struggle in interactive environments requiring long-horizon planning, primarily due to compounding errors when simulating future states. To address this, we propose ProAct, a fr…

Recurrent World Models Facilitate Policy Evolution

2018-09-04 · NeurIPS 2018 12 · David Ha, Jürgen Schmidhuber

A generative recurrent neural network is quickly trained in an unsupervised manner to model popular reinforcement learning environments through compressed spatio-temporal representations. The world model's extracted feat…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Dream to Drive with Predictive Individual World Model

2025-01-28 · Yinfeng Gao, Qichao Zhang, Da-wei Ding, Dongbin Zhao

It is still a challenging topic to make reactive driving behaviors in complex urban environments as road users' intentions are unknown. Model-based reinforcement learning (MBRL) offers great potential to learn a reactive…

Autonomous Driving model Model-based Reinforcement Learning reinforcement-learning+3

IANN-MPPI: Interaction-Aware Neural Network-Enhanced Model Predictive Path Integral Approach for Autonomous Driving

2025-07-16 · Kanghyun Ryu, Minjun Sung, Piyush Gupta, Jovin D'sa 외 arxiv

Motion planning for autonomous vehicles (AVs) in dense traffic is challenging, often leading to overly conservative behavior and unmet planning objectives. This challenge stems from the AVs' limited ability to anticipate…

Trajectory Planning Autonomous Vehicles Autonomous Driving Motion Planning