# A Versatile Multi-Agent Reinforcement Learning Benchmark for Inventory Management

**Authors:** Xianliang Yang, Zhihao Liu, Wei Jiang, Chuheng Zhang, Li Zhao, Lei Song, Jiang Bian  
**arXiv:** [2306.07542](https://arxiv.org/abs/2306.07542)  
**Date:** 2023-06-13  
**Summary page:** https://www.summarizepaper.com/en/paper/2306.07542v1/

## Key points

- MABIM is a multi-agent reinforcement learning (MARL) benchmark for inventory management
- MARL can be applied to various industrial scenarios such as autonomous driving, quantitative trading, and inventory management
- Applying MARL to real-world scenarios is impeded by challenges such as scaling up, complex agent interactions, and non-stationary dynamics
- MABIM is a multi-echelon, multi-commodity inventory management simulator that can generate versatile tasks with different challenging properties
- There is a lack of comprehensive benchmarks in the domain of inventory management despite extensive research conducted on this topic
- The authors provide an overview of existing efforts in this area and demonstrate how MABIM aligns more closely with real-world production scenarios while lending itself to be transformed into challenges for MARL algorithms effectively
- The paper introduces how the inventory management problem is modeled including the structure of the multi-echelon system, dynamic processes for each time step, and calculation of evaluation metrics such as profit
- Classic operations research (OR) methods and popular MARL algorithms are evaluated on challenging tasks using MABIM simulations to highlight their weaknesses and potential
- This study provides insights into how MARL can be applied to inventory management and the challenges that need to be addressed for successful implementation in real-world scenarios
- Overall, MABIM provides a valuable benchmark for researchers to develop and evaluate new MARL algorithms for inventory management.

## Summary

This paper introduces MABIM (Multi-Agent Benchmark for Inventory Management), a versatile multi-agent reinforcement learning (MARL) benchmark for inventory management. MARL models multiple agents that interact and learn within a shared environment, making it applicable to various industrial scenarios such as autonomous driving, quantitative trading, and inventory management. However, applying MARL to these real-world scenarios is impeded by many challenges such as scaling up, complex agent interactions, and non-stationary dynamics. To incentivize the research of MARL on these challenges in the context of inventory management, the authors develop MABIM which is a multi-echelon, multi-commodity inventory management simulator that can generate versatile tasks with different challenging properties. The paper also highlights the lack of comprehensive benchmarks in the domain of inventory management despite extensive research conducted on this topic. The authors provide an overview of existing efforts in this area and demonstrate how MABIM aligns more closely with real-world production scenarios while lending itself to be transformed into challenges for MARL algorithms effectively. In Section 3.1, the authors introduce how the inventory management problem is modeled in their paper including the structure of the multi-echelon system, dynamic processes for each time step, and calculation of evaluation metrics such as profit. Subsequently, they present the MARL formulation of this problem in Section 3.2. The multi-echelon model used in MABIM is motivated by real-world processes where products are produced by factories and transmitted through echelons of warehouses sequentially until they reach consumers. The goal is to optimize replenishment quantities for each restocking cycle or time step while balancing inventory to avoid overstocking or stockouts at any echelon level. Based on MABIM simulations, classic operations research (OR) methods and popular MARL algorithms are evaluated on challenging tasks to highlight their weaknesses and potential. This study provides insights into how MARL can be applied to inventory management and the challenges that need to be addressed for successful implementation in real-world scenarios. Overall, MABIM provides a valuable benchmark for researchers to develop and evaluate new MARL algorithms for inventory management.