Hierarchical Multi-Agent Multi-Armed Bandit for Resource Allocation in Multi-LEO Satellite Constellation Networks
Low Earth orbit (LEO) satellite constellation is capable of providing global coverage area with high-rate services in the next sixth-generation (6G) non-terrestrial network (NTN). Due to limited onboard resources of operating power, beams, and channels, resilient and efficient resource management has become compellingly imperative under complex interference cases. However, different from conventional terrestrial base stations, LEO is deployed at considerable height and under high mobility, inducing substantially long delay and interference during transmission. As a result, acquiring the accurate channel state information between LEOs and ground users is challenging. Therefore, we construct a framework with a two-way transmission under unknown channel information and no data collected at long-delay ground gateway. In this paper, we propose hierarchical multi-agent multi-armed bandit resource allocation for LEO constellation (mmRAL) by appropriately assigning available radio resources. LEOs are considered as collaborative multiple macro-agents attempting unknown trials of various actions of micro-agents of respective resources, asymptotically achieving suitable allocation with only throughput information. In simulations, we evaluate mmRAL in various cases of LEO deployment, serving numbers of users and LEOs, hardware cost and outage probability. Benefited by efficient and resilient allocation, the proposed mmRAL system is capable of operating in homogeneous or heterogeneous orbital planes or constellations, achieving the highest throughput performance compared to the existing benchmarks in open literature.
Code (0)
등록된 구현이 없습니다.
Tasks
ManagementMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Multi-armed Bandit Algorithm against Strategic Replication
We consider a multi-armed bandit problem in which a set of arms is registered by each agent, and the agent receives reward when its arm is selected. An agent might strategically submit more arms with replications, which …
Metadata-based Multi-Task Bandits with Bayesian Hierarchical Models
How to explore efficiently is a central problem in multi-armed bandits. In this paper, we introduce the metadata-based multi-task bandit problem, where the agent needs to solve a large number of related multi-armed bandi…
Multi-Armed BanditsThompson SamplingFair Algorithms for Multi-Agent Multi-Armed Bandits
We propose a multi-agent variant of the classical multi-armed bandit problem, in which there are $N$ agents and $K$ arms, and pulling an arm generates a (possibly different) stochastic reward for each agent. Unlike the c…
FairnessMulti-Armed BanditsCommunication-Efficient Collaborative Regret Minimization in Multi-Armed Bandits
In this paper, we study the collaborative learning model, which concerns the tradeoff between parallelism and communication overhead in multi-agent multi-armed bandits. For regret minimization in multi-armed bandits, we …
Multi-agent Reinforcement LearningMulti-Armed Banditsreinforcement-learningReinforcement Learning (RL)Using Subjective Logic to Estimate Uncertainty in Multi-Armed Bandit Problems
The multi-armed bandit problem is a classical decision-making problem where an agent has to learn an optimal action balancing exploration and exploitation. Properly managing this trade-off requires a correct assessment o…
Decision MakingMulti-Armed Bandits