Hierarchical Reinforcement Learning Framework for Stochastic Spaceflight Campaign Design

2021-03-16 · Yuji Takubo, Hao Chen, Koki Ho

This paper develops a hierarchical reinforcement learning architecture for multimission spaceflight campaign design under uncertainty, including vehicle design, infrastructure deployment planning, and space transportation scheduling. This problem involves a high-dimensional design space and is challenging especially with uncertainty present. To tackle this challenge, the developed framework has a hierarchical structure with reinforcement learning and network-based mixed-integer linear programming (MILP), where the former optimizes campaign-level decisions (e.g., design of the vehicle used throughout the campaign, destination demand assigned to each mission in the campaign), whereas the latter optimizes the detailed mission-level decisions (e.g., when to launch what from where to where). The framework is applied to a set of human lunar exploration campaign scenarios with uncertain in situ resource utilization performance as a case study. The main value of this work is its integration of the rapidly growing reinforcement learning research and the existing MILP-based space logistics methods through a hierarchical framework to handle the otherwise intractable complexity of space mission design under uncertainty. This unique framework is expected to be a critical steppingstone for the emerging research direction of artificial intelligence for space mission design.

📄 PDF Abstract BibTeX arXiv:2103.08981

Code (0)

등록된 구현이 없습니다.

Tasks

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Scheduling

Unveiling Parkinson's Disease-like Changes Triggered by Spaceflight

2024-08-25 · Nilufar Ali, Afshin Beheshti, Greg Hampikian

A meta-analysis of spaceflight data from both mouse and human flights reveals a striking overlap with Parkinson's disease (PD). Parallels include: changes in gait, loss of dopamine, sustained changes in the basal ganglia…

Hierarchical Capsule Prediction Network for Marketing Campaigns Effect

2022-08-22 · Zhixuan Chu, Hui Ding, Guang Zeng, Yuchen Huang 외

Marketing campaigns are a set of strategic activities that can promote a business's goal. The effect prediction for marketing campaigns in a real industrial scenario is very complex and challenging due to the fact that p…

Marketing Prediction

Functional Optimization Reinforcement Learning for Real-Time Bidding

2022-06-25 · Yining Lu, Changjie Lu, Naina Bandyopadhyay, Manoj Kumar 외

Real-time bidding is the new paradigm of programmatic advertising. An advertiser wants to make the intelligent choice of utilizing a \textbf{Demand-Side Platform} to improve the performance of their ad campaigns. Existin…

Attribute Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Mitigating Data Scarcity in Spaceflight Applications for Offline Reinforcement Learning Using Physics-Informed Deep Generative Models

2026-04-02 · Alex E. Ballentine, Nachiket U. Bapat, Raghvendra V. Cowlagi arxiv

The deployment of reinforcement learning (RL)-based controllers on physical systems is often limited by poor generalization to real-world scenarios, known as the simulation-to-reality (sim-to-real) gap. This gap is parti…

Synthetic Data Generation Reinforcement Learning Offline RL

Learning Multi-Timescale Abstractions for Hierarchical Combinatorial Planning

2026-05-16 · Vivienne Huiling Wang, Tinghuai Wang, Joni Pajarinen arxiv

The combination of exponentially large action spaces, stochastic dynamics, and long-horizon decision-making under limited resources makes Sequential Stochastic Combinatorial Optimization (SSCO) particularly challenging f…

Hierarchical Reinforcement Learning