Crossmodal Attentive Skill Learner

2017-11-28 · Shayegan Omidshafiei, Dong-Ki Kim, Jason Pazis, Jonathan P. How

This paper presents the Crossmodal Attentive Skill Learner (CASL), integrated with the recently-introduced Asynchronous Advantage Option-Critic (A2OC) architecture [Harb et al., 2017] to enable hierarchical reinforcement learning across multiple sensory inputs. We provide concrete examples where the approach not only improves performance in a single task, but accelerates transfer to new tasks. We demonstrate the attention mechanism anticipates and identifies useful latent features, while filtering irrelevant sensor modalities during execution. We modify the Arcade Learning Environment [Bellemare et al., 2013] to support audio queries, and conduct evaluations of crossmodal learning in the Atari 2600 game Amidar. Finally, building on the recent work of Babaeizadeh et al. [2017], we open-source a fast hybrid CPU-GPU implementation of CASL.

📄 PDF Abstract BibTeX arXiv:1711.10314

Code (1)

shayegano/CASL 공식 구현 tf

Tasks

Atari GamesCPUGPUHierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Case Relation Transformer: A Crossmodal Language Generation Model for Fetching Instructions

2021-07-02 · Motonari Kambara, Komei Sugiura

There have been many studies in robotics to improve the communication skills of domestic service robots. Most studies, however, have not fully benefited from recent advances in deep neural networks because the training d…

Relation Sentence Text Generation

A Bandit Approach to Posterior Dialog Orchestration Under a Budget

2019-06-22 · Sohini Upadhyay, Mayank Agarwal, Djallel Bounneffouf, Yasaman Khazaeni

Building multi-domain AI agents is a challenging task and an open problem in the area of AI. Within the domain of dialog, the ability to orchestrate multiple independently trained dialog agents, or skills, to create a un…

Priors, Hierarchy, and Information Asymmetry for Skill Transfer in Reinforcement Learning

2022-01-20 · Sasha Salter, Kristian Hartikainen, Walter Goodwin, Ingmar Posner

The ability to discover behaviours from past experience and transfer them to new tasks is a hallmark of intelligent agents acting sample-efficiently in the real world. Equipping embodied reinforcement learners with the s…

reinforcement-learning Reinforcement Learning (RL)Transfer Learning

A Simple Neural Attentive Meta-Learner

2017-07-11 · ICLR 2018 1 · Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, Pieter Abbeel

Deep neural networks excel in regimes with large amounts of data, but tend to struggle when data is scarce or when they need to adapt quickly to changes in the task. In response, recent work in meta-learning proposes tra…

Few-Shot Image Classification Meta-Learning Reinforcement Learning

The Modality Focusing Hypothesis: Towards Understanding Crossmodal Knowledge Distillation

2022-06-13 · Zihui Xue, Zhengqi Gao, Sucheng Ren, Hang Zhao

Crossmodal knowledge distillation (KD) extends traditional knowledge distillation to the area of multimodal learning and demonstrates great success in various applications. To achieve knowledge transfer across modalities…

Knowledge Distillation Transfer Learning