SplAgger: Split Aggregation for Meta-Reinforcement Learning

2024-03-05 · Jacob Beck, Matthew Jackson, Risto Vuorio, Zheng Xiong, Shimon Whiteson

A core ambition of reinforcement learning (RL) is the creation of agents capable of rapid learning in novel tasks. Meta-RL aims to achieve this by directly learning such agents. Black box methods do so by training off-the-shelf sequence models end-to-end. By contrast, task inference methods explicitly infer a posterior distribution over the unknown task, typically using distinct objectives and sequence models designed to enable task inference. Recent work has shown that task inference methods are not necessary for strong performance. However, it remains unclear whether task inference sequence models are beneficial even when task inference objectives are not. In this paper, we present evidence that task inference sequence models are indeed still beneficial. In particular, we investigate sequence models with permutation invariant aggregation, which exploit the fact that, due to the Markov property, the task posterior does not depend on the order of data. We empirically confirm the advantage of permutation invariant sequence models without the use of task inference objectives. However, we also find, surprisingly, that there are multiple conditions under which permutation variance remains useful. Therefore, we propose SplAgger, which uses both permutation variant and invariant components to achieve the best of both worlds, outperforming all baselines evaluated on continuous control and memory environments. Code is provided at https://github.com/jacooba/hyper.

📄 PDF Abstract BibTeX arXiv:2403.03020

Code (1)

jacooba/hyper 공식 구현 pytorch

Tasks

continuous-control Continuous ControlMeta Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Energy-efficient Deep Reinforcement Learning-based Network Function Disaggregation in Hybrid Non-terrestrial Open Radio Access Networks

2025-06-07 · S. M. Mahdi Shahabi, Xiaonan Deng, Ahmad Qidan, Taisir Elgorashi 외

This paper explores the integration of Open Radio Access Network (O-RAN) principles with non-terrestrial networks (NTN) and investigates the optimization of the functional split between Centralized Units (CU) and Distrib…

Deep Reinforcement Learning

Edge Learning via Federated Split Decision Transformers for Metaverse Resource Allocation

2026-02-18 · Fatih Temiz, Shavbo Salehi, Melike Erol-Kantarci arxiv

Mobile edge computing (MEC) based wireless metaverse services offer an untethered, immersive experience to users, where the superior quality of experience (QoE) needs to be achieved under stringent latency constraints an…

Reinforcement Learning Federated Learning Offline RL

PolicyGNN: Aggregation Optimization for Graph Neural Networks

2020-02-01

Graph data are pervasive in many real-world applications. Recently, increasing attention has been paid on graph neural networks (GNNs), which aim to model the local graph structures and capture the hierarchical patterns …

Deep Reinforcement Learning Reinforcement Learning (RL)

Policy-GNN: Aggregation Optimization for Graph Neural Networks

2020-06-26 · Kwei-Herng Lai, Daochen Zha, Kaixiong Zhou, Xia Hu

Deep Reinforcement Learning Node Classification Reinforcement Learning (RL)

Alternating Reinforcement Learning with Contextual Rubric Rewards: Beyond the Scalarization Strategy

2026-03-04 · Guangchen Lan, Lian Xiong, Xin Zhou, Hejie Cui 외 arxiv

Reinforcement Learning with Rubric Rewards (RLRR) is a framework that extends conventional reinforcement learning from human feedback (RLHF) and verifiable rewards (RLVR) by replacing scalar preference signals with struc…

Reinforcement Learning