Sim-to-Real Transfer in Deep Reinforcement Learning: Overcoming the „Simulation-Reality-Gap“ for Motion Control of an Unmanned Surface Vehicle

Unmanned surface vehicles (USVs) are boats that can navigate on the water surface without the
need for a human operator on board. Since there are many disturbances and nonlinear effects due to
the hydrodynamics of water, there is a need for robust motion control systems. Reinforcement
learning provides an approach to learn an optimal control policy that incorporates exploratory
interaction between an agent and an environment. In order to learn an optimal control strategy, a
large number of experiments is required. For this reason, training is often conducted in a simulation
environment, which differs from the real-world environment to some degree. This gap between
simulation and reality can lead to worse performance, or even failure, of the control policy when
deployed on the real system. Various approaches can be found in the literature based on transfer
learning and meta learning that enable fast adaption between different domains. However, to date,
there is no literature that addresses transfer for motion control of watercraft. This work is intended
to fill this gap by investigating the known methods for application to unmanned surface vehicles.