MetaTrader: An Reinforcement Learning Approach Integrating Diverse Policies for Portfolio Optimization
Portfolio management is a fundamental problem in finance. It involves periodic reallocations of assets to maximize the expected returns within an appropriate level of risk exposure. Deep reinforcement learning (RL) has been considered a promising approach to solving this problem owing to its strong capability in sequential decision making. However, due to the non-stationary nature of financial markets, applying RL techniques to portfolio optimization remains a challenging problem. Extracting trading knowledge from various expert strategies could be helpful for agents to accommodate the changing markets. In this paper, we propose MetaTrader, a novel two-stage RL-based approach for portfolio management, which learns to integrate diverse trading policies to adapt to various market conditions. In the first stage, MetaTrader incorporates an imitation learning objective into the reinforcement learning framework. Through imitating different expert demonstrations, MetaTrader acquires a set of trading policies with great diversity. In the second stage, MetaTrader learns a meta-policy to recognize the market conditions and decide on the most proper learned policy to follow. We evaluate the proposed approach on three real-world index datasets and compare it to state-of-the-art baselines. The empirical results demonstrate that MetaTrader significantly outperforms those baselines in balancing profits and risks. Furthermore, thorough ablation studies validate the effectiveness of the components in the proposed approach.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingDeep Reinforcement LearningImitation LearningManagementPortfolio Optimizationreinforcement-learningReinforcement Learning (RL)Sequential Decision MakingSimilar Papers 제목 키워드 기반
Your Offline Policy is Not Trustworthy: Bilevel Reinforcement Learning for Sequential Portfolio Optimization
Reinforcement learning (RL) has shown significant promise for sequential portfolio optimization tasks, such as stock trading, where the objective is to maximize cumulative returns while minimizing risks using historical …
Offline RLPortfolio OptimizationReinforcement Learning (RL)Stock PredictionExplainable Post hoc Portfolio Management Financial Policy of a Deep Reinforcement Learning agent
Financial portfolio management investment policies computed quantitatively by modern portfolio theory techniques like the Markowitz model rely on a set on assumptions that are not supported by data in high volatility mar…
Deep Reinforcement LearningFeature ImportanceManagementDeep Stock Trading: A Hierarchical Reinforcement Learning Framework for Portfolio Optimization and Order Execution
Portfolio management via reinforcement learning is at the forefront of fintech research, which explores how to optimally reallocate a fund into different financial assets over the long term by trial-and-error. Existing m…
Hierarchical Reinforcement LearningManagementPortfolio OptimizationReinforcement Learning (RL)Deep Reinforcement Learning and Mean-Variance Strategies for Responsible Portfolio Optimization
Portfolio optimization involves determining the optimal allocation of portfolio assets in order to maximize a given investment objective. Traditionally, some form of mean-variance optimization is used with the aim of max…
Deep Reinforcement LearningPortfolio Optimizationreinforcement-learningReinforcement LearningK-Myriad: Jump-starting reinforcement learning with unsupervised parallel agents
Parallelization in Reinforcement Learning is typically employed to speed up the training of a single policy, where multiple workers collect experience from an identical sampling distribution. This common design limits th…
Reinforcement LearningContinuous Control