Scalable Option Learning in High-Throughput Environments
Hierarchical reinforcement learning (RL) has the potential to enable effective decision-making over long timescales. Existing approaches, while promising, have yet to realize the benefits of large-scale training. In this work, we identify and solve several key challenges in scaling online hierarchical RL to high-throughput environments. We propose Scalable Option Learning (SOL), a highly scalable hierarchical RL algorithm which achieves a ~35x higher throughput compared to existing hierarchical methods. To demonstrate SOL's performance and scalability, we train hierarchical agents using 30 billion frames of experience on the complex game of NetHack, significantly surpassing flat agents and demonstrating positive scaling trends. We also validate SOL on MiniHack and Mujoco environments, showcasing its general applicability. Our code is open sourced at: github.com/facebookresearch/sol.
Code (0)
등록된 구현이 없습니다.
Tasks
Hierarchical Reinforcement LearningSimilar Papers 제목 키워드 기반
IMPACT: Importance Weighted Asynchronous Architectures with Clipped Target Networks
The practical usage of reinforcement learning agents is often bottlenecked by the duration of training time. To accelerate training, practitioners often turn to distributed reinforcement learning architectures to paralle…
continuous-controlContinuous Controlreinforcement-learningReinforcement Learning+1NAVIX: Scaling MiniGrid Environments with JAX
As Deep Reinforcement Learning (Deep RL) research moves towards solving large-scale worlds, efficient environment simulations become crucial for rapid experimentation. However, most existing environments struggle to scal…
CPUDeep Reinforcement LearningDiversityMeta-Learning+4HoverFast: an accurate, high-throughput, clinically deployable nuclear segmentation tool for brightfield digital pathology images
In computational digital pathology, accurate nuclear segmentation of Hematoxylin and Eosin (H&E) stained whole slide images (WSIs) is a critical step for many analyses and tissue characterizations. One popular deep learn…
GPUKnowledge DistillationNuclear SegmentationSegmentation+1Performance Characterization of Expert Router for Scalable LLM Inference
Large Language Models (LLMs) have experienced widespread adoption across scientific and industrial domains due to their versatility and utility for diverse tasks. Nevertheless, deploying and serving these models at scale…
Language ModellingAnonShield: Scalable On-Premise Pseudonymization for CSIRT Vulnerability Data
We present AnonShield, a high-throughput, on-premise pseudonymization system that combines GPU-accelerated NER, streaming processing, caching, and schema-aware configuration. Evaluated on datasets up to 550 MB (70,951 re…