MSPM: A Modularized and Scalable Multi-Agent Reinforcement Learning-based System for Financial Portfolio Management
Financial portfolio management (PM) is one of the most applicable problems in reinforcement learning (RL) owing to its sequential decision-making nature. However, existing RL-based approaches rarely focus on scalability or reusability to adapt to the ever-changing markets. These approaches are rigid and unscalable to accommodate the varying number of assets of portfolios and increasing need for heterogeneous data. Also, RL agents in the existing systems are ad-hoc trained and hardly reusable for different portfolios. To confront the above problems, a modular design is desired for the systems to be compatible with reusable asset-dedicated agents. In this paper, we propose a multi-agent RL-based system for PM (MSPM). MSPM involves two types of asynchronously-updated modules: Evolving Agent Module (EAM) and Strategic Agent Module (SAM). An EAM is an information-generating module with a DQN agent, and it receives heterogeneous data and generates signal-comprised information for a particular asset. An SAM is a decision-making module with a PPO agent for portfolio optimization, and it connects to EAMs to reallocate the assets in a portfolio. Trained EAMs can be connected to any SAM at will. With its modularized architecture, the multi-step condensation of volatile market information, and the reusable design of EAM, MSPM simultaneously addresses the two challenges in RL-based PM: scalability and reusability. Experiments on 8-year U.S. stock market data prove the effectiveness of MSPM in profit accumulation by its outperformance over five baselines in terms of accumulated rate of return (ARR), daily rate of return, and Sortino ratio. MSPM improves ARR by at least 186.5% compared to CRP, a widely-used PM strategy. To validate the indispensability of EAM, we back-test and compare MSPMs on four portfolios. EAM-enabled MSPMs improve ARR by at least 1341.8% compared to EAM-disabled MSPMs.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingManagementMulti-agent Reinforcement LearningPortfolio OptimizationReinforcement Learning (RL)Sequential Decision MakingTransfer LearningSimilar Papers 제목 키워드 기반
MSPM: A Multi-Site Physiological Monitoring Dataset for Remote Pulse, Respiration, and Blood Pressure Estimation
Visible-light cameras can capture subtle physiological biomarkers without physical contact with the subject. We present the Multi-Site Physiological Monitoring (MSPM) dataset, which is the first dataset collected to supp…
Blood pressure estimationMulti-Modal Spectral Parametrization Method (MMSPM) for analyzing EEG activity with distinct scaling regimes
Aperiodic neural activity has been the subject of intense research interest lately as it could reflect on the cortical excitation/inhibition ratio, which is suspected to be affected in numerous clinical conditions. This …
EEGPhysics-informed Modularized Neural Network for Advanced Building Control by Deep Reinforcement Learning
Physics-informed machine learning (PIML) provides a promising solution for building energy modeling and can serve as a virtual environment to enable reinforcement learning (RL) agents to interact and learn. However, chal…
Deep Reinforcement LearningPhysics-informed machine learningReinforcement Learning (RL)A Multi-Objective Deep Reinforcement Learning Framework
This paper introduces a new scalable multi-objective deep reinforcement learning (MODRL) framework based on deep Q-networks. We develop a high-performance MODRL framework that supports both single-policy and multi-policy…
Deep Reinforcement LearningMulti-Objective Reinforcement Learningreinforcement-learningReinforcement Learning+1A multi-scale lithium-ion battery capacity prediction using mixture of experts and patch-based MLP
Lithium-ion battery health management has become increasingly important as the application of batteries expands. Precise forecasting of capacity degradation is critical for ensuring the healthy usage of batteries. In thi…
Mixture-of-Experts