Vision Language Model-based Testing of Industrial Autonomous Mobile Robots
PAL Robotics, in Spain, builds a variety of Autonomous Mobile Robots (AMRs), which are deployed in diverse environments (e.g., warehouses, retail spaces, and offices), where they work alongside humans. Given that human behavior can be unpredictable and that AMRs may not have been trained to handle all possible unknown and uncertain behaviors, it is important to test AMRs under a wide range of human interactions to ensure their safe behavior. Moreover, testing in real environments with actual AMRs and humans is often costly, impractical, and potentially hazardous (e.g., it could result in human injury). To this end, we propose a Vision Language Model (VLM)-based testing approach (RVSG) for industrial AMRs developed together with PAL Robotics. Based on the functional and safety requirements, RVSG uses the VLM to generate diverse human behaviors that violate these requirements. We evaluated RVSG with several requirements and navigation routes in a simulator using the latest AMR from PAL Robotics. Our results show that, compared with the baseline, RVSG can effectively generate requirement-violating scenarios. Moreover, RVSG-generated scenarios increase variability in robot behavior, thereby helping reveal their uncertain behaviors.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
MobileAgentBench: An Efficient and User-Friendly Benchmark for Mobile LLM Agents
Large language model (LLM)-based mobile agents are increasingly popular due to their capability to interact directly with mobile phone Graphic User Interfaces (GUIs) and their potential to autonomously manage daily tasks…
BenchmarkingLanguage ModelingLanguage ModellingLarge Language ModelInterference Aware Path Planning for Mobile Robots in mmWave Multi Cell Networks
The emerging beyond 5G and envisioned 6G wireless networks are considered as key enablers in supporting a diversified set of applications for industrial mobile robots (MRs). The scenario under investigation in this paper…
Vision-based Manipulation of Transparent Plastic Bags in Industrial Setups
This paper addresses the challenges of vision-based manipulation for autonomous cutting and unpacking of transparent plastic bags in industrial setups, aligning with the Industry 4.0 paradigm. Industry 4.0, driven by dat…
Slug Mobile: Test-Bench for RL Testing
Sim-to real gap in Reinforcement Learning is when a model trained in a simulator does not translate to the real world. This is a problem for Autonomous Vehicles (AVs) as vehicle dynamics can vary from simulation to reali…
Autonomous VehiclesMonocular visual autonomous landing system for quadcopter drones using software in the loop
Autonomous landing is a capability that is essential to achieve the full potential of multi-rotor drones in many social and industrial applications. The implementation and testing of this capability on physical platforms…