Sim-to-Real Transfer in Deep Reinforcement Learning: Overcoming the „Simulation-Reality-Gap“ for Motion Control of an Unmanned Surface Vehicle

Unmanned surface vehicles (USVs) are boats that can navigate on the water surface without the need for a human operator on board. Since there are many isturbances and nonlinear effects due to the hydrodynamics of water, there is a need for robust motion control systems. Reinforcement learning provides an approach to learn an optimal control policy that incorporates exploratory interaction between an agent and an environment. In order to learn an optimal control strategy, a large number of experiments is required. For this reason, training is often conducted in a simulation environment, which differs from the real-world environment to some degree. This gap between simulation and reality can lead to worse performance, or even failure, of the control policy when deployed on the real system. Various approaches can be found in the literature based on transfer learning and meta learning that enable fast adaption between different domains. However, to date, there is no literature that addresses transfer for motion control of watercraft. This work is intended to fill this gap by investigating the known methods for application to unmanned surface vehicles.