SegDAC: Visual Generalization in Reinforcement Learning via Dynamic Object Tokens

2025-08-12 · Alexandre Brown, Glen Berseth arxiv

Visual reinforcement learning policies trained on pixel observations often struggle to generalize when visual conditions change at test time. Object-centric representations are a promising alternative, but most approaches use fixed-size slot representations, require image reconstruction, or need auxiliary losses to learn object decompositions. As a result, it remains unclear how to learn RL policies directly from object-level inputs without these constraints. We propose SegDAC, a Segmentation-Driven Actor-Critic that operates on a variable-length set of object token embeddings. At each timestep, text-grounded segmentation produces object masks from which spatially aware token embeddings are extracted. A transformer-based actor-critic processes these dynamic tokens, using segment positional encoding to preserve spatial information across objects. We ablate these design choices and show that both segment positional encoding and variable-length processing are individually necessary for strong performance. We evaluate SegDAC on 8 ManiSkill3 manipulation tasks under 12 visual perturbation types across 3 difficulty levels. SegDAC improves over prior visual generalization methods by 15% on easy, 66% on medium, and 88% on the hardest settings. SegDAC matches the sample efficiency of the state-of-the-art visual RL methods while achieving improved generalization under visual changes. Project Page: https://segdac.github.io/

📄 PDF Abstract BibTeX arXiv:2508.09325

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning Image Reconstruction

Environment Agnostic Representation for Visual Reinforcement Learning

2023-01-01 · ICCV 2023 1 · Hyesong Choi, Hunsang Lee, Seongwon Jeong, Dongbo Min

Generalization capability of vision-based deep reinforcement learning (RL) is indispensable to deal with dynamic environment changes that exist in visual observations. The high-dimensional space of the visual input, …

Deep Reinforcement Learning Domain Generalization reinforcement-learning Reinforcement Learning+1

Object-Oriented Dynamics Predictor

2018-05-25 · NeurIPS 2018 12 · Guangxiang Zhu, Zhiao Huang, Chongjie Zhang

Generalization has been one of the major challenges for learning dynamics models in model-based reinforcement learning. However, previous work on action-conditioned dynamics prediction focuses on learning the pixel-level…

Model-based Reinforcement Learning Object Reinforcement Learning

Saliency-Guided Representation with Consistency Policy Learning for Visual Unsupervised Reinforcement Learning

2026-04-07 · Jingbo Sun, Qichao Zhang, Songjun Tu, Xing Fang 외 arxiv

Zero-shot unsupervised reinforcement learning (URL) offers a promising direction for building generalist agents capable of generalizing to unseen tasks without additional supervision. Among existing approaches, successor…

Zero-shot Generalization Representation Learning Reinforcement Learning

Visual Grounding for Object-Level Generalization in Reinforcement Learning

2024-08-04 · Haobin Jiang, Zongqing Lu

Generalization is a pivotal challenge for agents following natural language instructions. To approach this goal, we leverage a vision-language model (VLM) for visual grounding and transfer its vision-language knowledge i…

Language Modelling Object reinforcement-learning Reinforcement Learning+3

Generalization in Visual Reinforcement Learning with the Reward Sequence Distribution

2023-02-19 · Jie Wang, Rui Yang, Zijie Geng, Zhihao Shi 외

Generalization in partially observed markov decision processes (POMDPs) is critical for successful applications of visual reinforcement learning (VRL) in real scenarios. A widely used idea is to learn task-relevant repre…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Representation Learning