Deep Reinforcement Learning for Path-Following Control of an Autonomous Surface Vehicle using Domain Randomization

Abstract :

In this paper, we propose a path-following controller for an autonomous surface vehicle (ASV) that is based on model-free deep reinforcement learning. To make the learning agent more robust, we investigate domain randomization for sim-to-real transfer. We provide a comparison between three different algorithms: Deep Deterministic Policy Gradient (DDPG), Soft Actor-Critic (SAC) and Proximal Policy Optimization (PPO). The trained models are evaluated on the small-scale ASV Altus-LSA Niriis in the maritime test basin at DFKI RIC, Germany. Our results show that applying domain randomization leads to a significant performance improvement compared to no domain randomization, when tested on real hardware.

Keywords :

Unmanned marine vehicles, Reinforcement learning control, Trajectory Tracking and Path Following, Sim-to-Real Transfer, Domain Randomization