2-Level Reinforcement Learning for Ships on Inland Waterways: Path Planning and Following

2023-07-25 · Martin Waltz, Niklas Paulig, Ostap Okhrin

This paper proposes a realistic modularized framework for controlling autonomous surface vehicles (ASVs) on inland waterways (IWs) based on deep reinforcement learning (DRL). The framework improves operational safety and comprises two levels: a high-level local path planning (LPP) unit and a low-level path following (PF) unit, each consisting of a DRL agent. The LPP agent is responsible for planning a path under consideration of dynamic vessels, closing a gap in the current research landscape. In addition, the LPP agent adequately considers traffic rules and the geometry of the waterway. We thereby introduce a novel application of a spatial-temporal recurrent neural network architecture to continuous action spaces. The LPP agent outperforms a state-of-the-art artificial potential field (APF) method by increasing the minimum distance to other vessels by 65% on average. The PF agent performs low-level actuator control while accounting for shallow water influences and the environmental forces winds, waves, and currents. Compared with a proportional-integral-derivative (PID) controller, the PF agent yields only 61% of the mean cross-track error (MCTE) while significantly reducing control effort (CE) in terms of the required absolute rudder angle. Lastly, both agents are jointly validated in simulation, employing the lower Elbe in northern Germany as an example case and using real automatic identification system (AIS) trajectories to model the behavior of other ships.

📄 PDF Abstract BibTeX arXiv:2307.16769

Code (1)

marwaltz/tud_rl 공식 구현 pytorch

Tasks

Deep Reinforcement Learningreinforcement-learning

Distributed MPC for autonomous ships on inland waterways with collaborative collision avoidance

2024-03-01 · Hoang Anh Tran, Tor Arne Johansen, Rudy R. Negenborn

This paper presents a distributed solution for the problem of collaborative collision avoidance for autonomous inland waterway ships. A two-layer collision avoidance framework that considers inland waterway traffic regul…

Collision Avoidance Model Predictive Control

Safety Aware Autonomous Path Planning Using Model Predictive Reinforcement Learning for Inland Waterways

2023-11-16 · Astrid Vanneste, Simon Vanneste, Olivier Vasseur, Robin Janssens 외

In recent years, interest in autonomous shipping in urban waterways has increased significantly due to the trend of keeping cars and trucks out of city centers. Classical approaches such as Frenet frame based planning an…

Navigate reinforcement-learning Reinforcement Learning

Robust Path Following on Rivers Using Bootstrapped Reinforcement Learning

2023-03-24 · Niklas Paulig, Ostap Ohkrin

This paper develops a Deep Reinforcement Learning (DRL)-agent for navigation and control of autonomous surface vessels (ASV) on inland waterways. Spatial restrictions due to waterway geometry and the resulting challenges…

Deep Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning

Safe Robust Predictive Control-based Motion Planning of Automated Surface Vessels in Inland Waterways

2025-09-08 · Sajad Ahmadi, Hossein Nejatbakhsh Esfahani, Javad Mohammadpour Velni arxiv

Deploying self-navigating surface vessels in inland waterways offers a sustainable alternative to reduce road traffic congestion and emissions. However, navigating confined waterways presents unique challenges, including…

Collision Avoidance Motion Planning

Towards Explainable Deep Learning for Ship Trajectory Prediction in Inland Waterways

2026-03-04 · Tom Legel, Dirk Söffker, Roland Schätzle, Kathrin Donandt arxiv

Accurate predictions of ship trajectories in crowded environments are essential to ensure safety in inland waterways traffic. Recent advances in deep learning promise increased accuracy even for complex scenarios. While …

Trajectory Prediction