CIC: Contrastive Intrinsic Control for Unsupervised Skill Discovery

2022-02-01 · Michael Laskin, Hao liu, Xue Bin Peng, Denis Yarats, Aravind Rajeswaran, Pieter Abbeel

We introduce Contrastive Intrinsic Control (CIC), an algorithm for unsupervised skill discovery that maximizes the mutual information between state-transitions and latent skill vectors. CIC utilizes contrastive learning between state-transitions and skills to learn behavior embeddings and maximizes the entropy of these embeddings as an intrinsic reward to encourage behavioral diversity. We evaluate our algorithm on the Unsupervised Reinforcement Learning Benchmark, which consists of a long reward-free pre-training phase followed by a short adaptation phase to downstream tasks with extrinsic rewards. CIC substantially improves over prior methods in terms of adaptation efficiency, outperforming prior unsupervised skill discovery methods by 1.79x and the next leading overall exploration algorithm by 1.18x.

📄 PDF Abstract BibTeX arXiv:2202.00161

Code (1)

rll-research/cic pytorch

Tasks

Contrastive LearningDiversityreinforcement-learning Reinforcement Learning (RL)Unsupervised Reinforcement Learning

Methods 이 논문이 사용한 방법론

Contrastive Learning 설명 없음

Behavior Contrastive Learning for Unsupervised Skill Discovery

2023-05-08 · Rushuai Yang, Chenjia Bai, Hongyi Guo, Siyuan Li 외

In reinforcement learning, unsupervised skill discovery aims to learn diverse skills without extrinsic rewards. Previous methods discover skills by maximizing the mutual information (MI) between states and skills. Howeve…

continuous-control Continuous Control Contrastive Learning

ComSD: Balancing Behavioral Quality and Diversity in Unsupervised Skill Discovery

2023-09-29 · Xin Liu, Yaran Chen, Dongbin Zhao

This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. Unsupervised skill discovery seeks to acquire differen…

Contrastive Learning Diversity Reinforcement Learning (RL)Unsupervised Reinforcement Learning

Controllability-Aware Unsupervised Skill Discovery

2023-02-10 · Seohong Park, Kimin Lee, Youngwoon Lee, Pieter Abbeel

One of the key capabilities of intelligent agents is the ability to discover useful skills without external supervision. However, the current unsupervised skill discovery methods are often limited to acquiring simple, ea…

SUSD: Structured Unsupervised Skill Discovery through State Factorization

2026-02-02 · Seyed Mohammad Hadi Hosseini, Mahdieh Soleymani Baghshah arxiv

Unsupervised Skill Discovery (USD) aims to autonomously learn a diverse set of skills without relying on extrinsic rewards. One of the most common USD approaches is to maximize the Mutual Information (MI) between skill l…

Hierarchical Reinforcement Learning

Constrained Intrinsic Motivation for Reinforcement Learning

2024-07-12 · Xiang Zheng, Xingjun Ma, Chao Shen, Cong Wang

This paper investigates two fundamental problems that arise when utilizing Intrinsic Motivation (IM) for reinforcement learning in Reward-Free Pre-Training (RFPT) tasks and Exploration with Intrinsic Motivation (EIM) tas…

MuJoCo reinforcement-learning Reinforcement Learning