Gimitest: A Comprehensive Tool for Testing Reinforcement Learning Policies

2026-07-08 · Dennis Gross, Quentin Mazouni, Helge Spieker, Arnaud Gotlieb arxiv

Reinforcement learning (RL) policies can be unsafe and vulnerable to attacks. Ensuring their reliability is often a pain point as existing automated testing methods target only selected environments, testing scenarios, and RL algorithms. To address this, we propose a comprehensive framework for testing single- and multi-agent RL policies under varying conditions. Our implementation of this framework, Gimitest, is an open-source tool that supports various gym frameworks and allows for modifications of their integrated components. This article describes the framework and details Gimitest's functionality and architecture. It showcases its effectiveness in testing multiple RL policies in environments such as the official Farama Gymnasium and PettingZoo.

📄 PDF Abstract BibTeX arXiv:2607.07029

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

COOL-MC: A Comprehensive Tool for Reinforcement Learning and Model Checking

2022-09-15 · Dennis Gross, Nils Jansen, Sebastian Junges, Guillermo A. Perez

This paper presents COOL-MC, a tool that integrates state-of-the-art reinforcement learning (RL) and model checking. Specifically, the tool builds upon the OpenAI gym and the probabilistic model checker Storm. COOL-MC pr…

OpenAI Gym reinforcement-learning Reinforcement Learning (RL)

MOLTE: a Modular Optimal Learning Testing Environment

2017-09-13 · Yingfei Wang, Warren Powell

We address the relative paucity of empirical testing of learning algorithms (of any type) by introducing a new public-domain, Modular, Optimal Learning Testing Environment (MOLTE) for Bayesian ranking and selection probl…

Experimental Design

Towards Quadrupedal Jumping and Walking for Dynamic Locomotion using Reinforcement Learning

2025-10-28 · Jørgen Anker Olsen, Lars Rønhaug Pettersen, Kostas Alexis arxiv

This paper presents a curriculum-based reinforcement learning framework for training precise and high-performance jumping policies for the robot `Olympus'. Separate policies are developed for vertical and horizontal jump…

Reinforcement Learning

Policy Design for Active Sequential Hypothesis Testing using Deep Learning

2018-10-11 · Dhruva Kartik, Ekraam Sabir, Urbashi Mitra, Prem Natarajan

Information theory has been very successful in obtaining performance limits for various problems such as communication, compression and hypothesis testing. Likewise, stochastic control theory provides a characterization …

Deep Learning Deep Reinforcement Learning Reinforcement Learning Two-sample testing

An Analysis of Switchback Designs in Reinforcement Learning

2024-03-26 · Qianglin Wen, Chengchun Shi, Ying Yang, Niansheng Tang 외

This paper offers a detailed investigation of switchback designs in A/B testing, which alternate between baseline and new policies over time. Our aim is to thoroughly evaluate the effects of these designs on the accuracy…

reinforcement-learning Reinforcement Learning