Comparing Deep Reinforcement Learning Algorithms in Two-Echelon Supply Chains

2022-04-20 · Francesco Stranieri, Fabio Stella

In this study, we analyze and compare the performance of state-of-the-art deep reinforcement learning algorithms for solving the supply chain inventory management problem. This complex sequential decision-making problem consists of determining the optimal quantity of products to be produced and shipped across different warehouses over a given time horizon. In particular, we present a mathematical formulation of a two-echelon supply chain environment with stochastic and seasonal demand, which allows managing an arbitrary number of warehouses and product types. Through a rich set of numerical experiments, we compare the performance of different deep reinforcement learning algorithms under various supply chain structures, topologies, demands, capacities, and costs. The results of the experimental plan indicate that deep reinforcement learning algorithms outperform traditional inventory management strategies, such as the static (s, Q)-policy. Furthermore, this study provides detailed insight into the design and development of an open-source software library that provides a customizable environment for solving the supply chain inventory management problem using a wide range of data-driven approaches.

📄 PDF Abstract BibTeX arXiv:2204.09603

Code (1)

frenkowski/scimai-gym 공식 구현

Tasks

Decision MakingDeep Reinforcement LearningManagement reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Sequential Decision Making

Methods 이 논문이 사용한 방법론

Entropy Regularization 설명 없음

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

A3C A3C, Asynchronous Advantage Actor Critic, is a policy gradient algorithm in reinforcement learning that maintains a policy $\pi\left(a\_{t}\mid{s}\_{t}; \theta\right)$ and…

PPO Proximal Policy Optimization, or PPO, is a policy gradient method for reinforcement learning. The motivation was to have an algorithm with the data efficiency and reliable…

Stock-out Prediction in Multi-echelon Networks

2017-09-20 · Afshin Oroojlooyjadid, Lawrence Snyder, Martin Takáč

In multi-echelon inventory systems the performance of a given node is affected by events that occur at many other nodes and in many other time periods. For example, a supply disruption upstream will have an effect on dow…

Prediction

Interpretable Reinforcement Learning via Neural Additive Models for Inventory Management

2023-03-18 · Julien Siems, Maximilian Schambach, Sebastian Schulze, Johannes S. Otterbach

The COVID-19 pandemic has highlighted the importance of supply chains and the role of digital management to react to dynamic changes in the environment. In this work, we focus on developing dynamic inventory ordering pol…

Additive models Management reinforcement-learning Reinforcement Learning+1

Partition-form Cooperative Games in Two-Echelon Supply Chains

2024-01-10 · Gurkirat Wadhwa, Tushar Shankar Walunj, Veeraruna Kavitha

Competition and cooperation are inherent features of any multi-echelon supply chain. The interactions among the agents across the same echelon and that across various echelons influence the percolation of market demand a…

Form

Simultaneous Decision Making for Stochastic Multi-echelon Inventory Optimization with Deep Neural Networks as Decision Makers

2020-06-10 · Mohammad Pirhooshyaran, Lawrence V. Snyder

We propose a framework that uses deep neural networks (DNN) to optimize inventory decisions in complex multi-echelon supply chains. We first introduce pairwise modeling of general stochastic multi-echelon inventory optim…

Decision Making

Multi-echelon Supply Chains with Uncertain Seasonal Demands and Lead Times Using Deep Reinforcement Learning

2022-01-12 · Julio César Alves, Geraldo Robson Mateus

We address the problem of production planning and distribution in multi-echelon supply chains. We consider uncertain demands and lead times which makes the problem stochastic and non-linear. A Markov Decision Process for…

Decision Making Deep Reinforcement Learning Reinforcement Learning (RL)Sequential Decision Making