When using reinforcement learning, simulations are essential, because the agent can learn without the risk of damaging the hardware in a real-world experiment. This is particularly evident in the domain of underwater robotics, where real-world experiments and failures are expensive. Following the training of an agent on a simulator, a simulation-to-reality gap remains, because a simulation cannot be an accurate representation of the real world. Closing this gap is important to improve the performance of the model once it is transferred to the real robot.
The objective of this master's thesis is to determine how the design of the simulation can influence the simulation-to-reality gap and which parts of the simulation are most important for reducing this gap. The reinforcement learning algorithm is tasked with exploring an unknown environment and counting objects. In order to assess the performance of different models, both the score in the task and the computational cost during training are taken into account.