Pretraining for Language Conditioned Imitation with Transformers

2021-09-29 · Aaron L Putterman, Kevin Lu, Igor Mordatch, Pieter Abbeel

We study reinforcement learning (RL) agents which can utilize language inputs. To investigate this, we propose a new multimodal benchmark -- Text-Conditioned Frostbite -- in which an agent must complete tasks specified by text instructions in the Atari Frostbite environment. We curate and release a dataset of 5M text-labelled transitions for training and to encourage further research in this direction. On this benchmark, we evaluate Text Decision Transformer (TDT), a transformer directly operating on text, state, and action tokens, and find it improves upon other baseline architectures. Furthermore, we evaluate the effect of pretraining, finding unsupervised pretraining can yield improved results in low-data settings.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Residual Connection 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

FactoFormer: Factorized Hyperspectral Transformers with Self-Supervised Pretraining

2023-09-18 · Shaheer Mohamed, Maryam Haghighat, Tharindu Fernando, Sridha Sridharan 외

Hyperspectral images (HSIs) contain rich spectral and spatial information. Motivated by the success of transformers in the field of natural language processing and computer vision where they have shown the ability to lea…

From Zero to Hero: On the Limitations of Zero-Shot Cross-Lingual Transfer with Multilingual Transformers

2020-05-01 · Anne Lauscher, Vinit Ravishankar, Ivan Vulić, Goran Glavaš

Massively multilingual transformers pretrained with language modeling objectives (e.g., mBERT, XLM-R) have become a de facto default transfer paradigm for zero-shot cross-lingual transfer in NLP, offering unmatched trans…

Cross-Lingual Transfer Cross-Lingual Word Embeddings Dependency Parsing Language Modeling+7

From Zero to Hero: On the Limitations of Zero-Shot Language Transfer with Multilingual Transformers

2020-11-01 · EMNLP 2020 11 · Anne Lauscher, Vinit Ravishankar, Ivan Vuli{\'c}, Goran Glava{\v{s}}

Massively multilingual transformers (MMTs) pretrained via language modeling (e.g., mBERT, XLM-R) have become a default paradigm for zero-shot language transfer in NLP, offering unmatched transfer performance. Current eva…

Cross-Lingual Word Embeddings Dependency Parsing Language Modeling Language Modelling+5

Attributing Culture-Conditioned Generations to Pretraining Corpora

2024-12-30 · Huihan Li, Arnav Goel, Keyu He, Xiang Ren

In open-ended generative tasks like narrative writing or dialogue, large language models often exhibit cultural biases, showing limited knowledge and generating templated outputs for less prevalent cultures. Recent works…

Memorization

In-Context Curiosity: Distilling Exploration for Decision-Pretrained Transformers on Bandit Tasks

2025-09-30 · Huitao Yang, Guanting Chen arxiv

As large language models (LLMs) continue to grow in capability, there is increasing interest in incorporating them into decision-making tasks. A common pipeline for this is Decision-Pretrained Transformers (DPTs). Howeve…

Multi-Armed Bandits