What can I do here? A Theory of Affordances in Reinforcement Learning

2020-06-26 · ICML 2020 1 · Khimya Khetarpal, Zafarali Ahmed, Gheorghe Comanici, David Abel, Doina Precup

Reinforcement learning algorithms usually assume that all actions are always available to an agent. However, both people and animals understand the general link between the features of their environment and the actions that are feasible. Gibson (1977) coined the term "affordances" to describe the fact that certain states enable an agent to do certain actions, in the context of embodied agents. In this paper, we develop a theory of affordances for agents who learn and plan in Markov Decision Processes. Affordances play a dual role in this case. On one hand, they allow faster planning, by reducing the number of actions available in any given situation. On the other hand, they facilitate more efficient and precise learning of transition models from data, especially when such models require function approximation. We establish these properties through theoretical results as well as illustrative examples. We also propose an approach to learn affordances and use it to estimate transition models that are simpler and generalize better.

📄 PDF Abstract BibTeX arXiv:2006.15085

Code (1)

deepmind/affordances_option_models 공식 구현

Tasks

reinforcement-learning Reinforcement Learning (RL)

Rediscovering Affordance: A Reinforcement Learning Perspective

2021-12-24 · Yi-Chi Liao, Kashyap Todi, Aditya Acharya, Antti Keurulainen 외

Affordance refers to the perception of possible actions allowed by an object. Despite its relevance to human-computer interaction, no existing theory explains the mechanisms that underpin affordance-formation; that is, h…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Possibility Before Utility: Learning And Using Hierarchical Affordances

2022-03-23 · ICLR 2022 4 · Robby Costales, Shariq Iqbal, Fei Sha

Reinforcement learning algorithms struggle on tasks with complex hierarchical dependency structures. Humans and other intelligent agents do not waste time assessing the utility of every high-level action in existence, bu…

Hierarchical Reinforcement Learning Navigate reinforcement-learning Reinforcement Learning+1

What is it Like to Be a Bot: Simulated, Situated, Structurally Coherent Qualia (S3Q) Theory of Consciousness

2021-03-13 · K. Schmidt, J. Culbertson, C. Cox, H. S. Clouse 외

A novel representationalist theory of consciousness is presented that is grounded in neuroscience and provides a path to artificially conscious computing. Central to the theory are representational affordances of the con…

2D Human Pose Estimation 2D Object Detection 2D Semantic Segmentation 2D Semantic Segmentation task 1 (8 classes)+1

Learning at the Ends: From Hand to Tool Affordances in Humanoid Robots

2018-04-09 · Giovanni Saponaro, Pedro Vicente, Atabak Dehban, Lorenzo Jamone 외

One of the open challenges in designing robots that operate successfully in the unpredictable human environment is how to make them able to predict what actions they can perform on objects, and what their effects will be…

Decision Making

Learning Intermediate Features of Object Affordances with a Convolutional Neural Network

2020-02-20 · Aria Yuan Wang, Michael J. Tarr

Our ability to interact with the world around us relies on being able to infer what actions objects afford -- often referred to as affordances. The neural mechanisms of object-action associations are realized in the visu…