OFCOURSE: A Multi-Agent Reinforcement Learning Environment for Order Fulfillment

2023-09-26 · NeurIPS 2023 11

The dramatic growth of global e-commerce has led to a surge in demand for efficient and cost-effective order fulfillment which can increase customers' service levels and sellers' competitiveness. However, managing order fulfillment is challenging due to a series of interdependent online sequential decision-making problems. To clear this hurdle, rather than solving the problems separately as attempted in some recent researches, this paper proposes a method based on multi-agent reinforcement learning to integratively solve the series of interconnected problems, encompassing order handling, packing and pickup, storage, order consolidation, and last-mile delivery. In particular, we model the integrated problem as a Markov game, wherein a team of agents learns a joint policy via interacting with a simulated environment. Since no simulated environment supporting the complete order fulfillment problem exists, we devise Order Fulfillment COoperative mUlti-agent Reinforcement learning Scalable Environment (OFCOURSE) in the OpenAI Gym style, which allows reproduction and re-utilization to build customized applications. By constructing the fulfillment system in OFCOURSE, we optimize a joint policy that solves the integrated problem, facilitating sequential order-wise operations across all fulfillment units and minimizing the total cost of fulfilling all orders within the promised time. With OFCOURSE, we also demonstrate that the joint policy learned by multi-agent reinforcement learning outperforms the combination of locally optimal policies. The source code of OFCOURSE is available at: https://github.com/GitYiheng/ofcourse.

📄 PDF Abstract BibTeX

Code (1)

gityiheng/ofcourse 공식 구현 pytorch

Methods 이 논문이 사용한 방법론

Golden Queue Managers 설명 없음

Optimizing Market Making using Multi-Agent Reinforcement Learning

2018-12-26 · Yagna Patel

In this paper, reinforcement learning is applied to the problem of optimizing market making. A multi-agent reinforcement learning framework is used to optimally place limit orders that lead to successful trades. The fram…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Multi-Agent Reinforcement Learning in a Realistic Limit Order Book Market Simulation

2020-06-10 · Michaël Karpe, Jin Fang, Zhongyao Ma, Chen Wang

Optimal order execution is widely studied by industry practitioners and academic researchers because it determines the profitability of investment decisions and high-level trading strategies, particularly those involving…

Multi-agent Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning (RL)

C-COMA: A CONTINUAL REINFORCEMENT LEARNING MODEL FOR DYNAMIC MULTIAGENT ENVIRONMENTS

2021-04-05 · Kyueyeol Jung, Incheol Kim

It is very important to learn behavioral policies that allow multiple agents to work together organically for common goals in various real-world applications. In this multi-agent reinforcement learning (MARL) environment…

Continual Learning Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+3

Multi-Agent Reinforcement Learning for Order-dispatching via Order-Vehicle Distribution Matching

2019-10-07 · Ming Zhou, Jiarui Jin, Wei-Nan Zhang, Zhiwei Qin 외

Improving the efficiency of dispatching orders to vehicles is a research hotspot in online ride-hailing systems. Most of the existing solutions for order-dispatching are centralized controlling, which require to consider…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Reinforcement Learning-Based Market Making as a Stochastic Control on Non-Stationary Limit Order Book Dynamics

2025-09-15 · Rafael Zimmer, Oswaldo Luiz do Valle Costa arxiv

Reinforcement Learning has emerged as a promising framework for developing adaptive and data-driven strategies, enabling market makers to optimize decision-making policies based on interactions with the limit order book …

Reinforcement Learning