VeryHuman

Biologically inspired control algorithms for robots have been proven very successful. Often, these algorithms use techniques such as reinforcement learning or optimal control to perform sophisticated movement patterns with a robot (for e.g. humanoid walking). However, two main challenges exist for these learning-based approaches:

• A robust hardware of the robot along with an accurate simulation of the system is required. For example, the robot can be subjected to a large amount of holonomic constraints including internal closed loops and external contacts which pose challenges to the accuracy of the simulation.
• Second, control algorithms of this kind can be hard to implement due to lack of knowledge of reward and constraints. As an example, consider the upright walking movement for a two-legged humanoid robot. It is not immediately clear how one can specify the task of “upright walking”. We might try to relate different body parts (head above shoulders, shoulders above waist, waist above legs), use physical stability criteria (centre of pressure, zero moment point etc), but do these really specify walking and what are non-trivial properties? This leads to the non-trivial task of defining a suitable reward function for (deep) reinforcement learning approaches or cost function for optimal control approaches along with constraints.

This project aims at three basic research questions:

• How can we formulate and prove properties of a complex humanoid robot, and
• how can we efficiently combine reinforcement learning and optimal control-based approaches, and
• how can we make use of symbolic properties to derive a reward function in a deep reinforcement learning approach or optimal control approach for complex use cases such as humanoid walking.

These three research questions are closely interwoven and are dealt with in three work areas that lead to the overarching goal of the project: a methodology to develop a hybrid of deep (reinforcement) learning and optimization based control of a robot together with a corresponding rational reconstruction of its observed and future behavior. This reconstruction is based on observations of the robot’s movements and general knowledge of physics (rigid body dynamics). The overall approach is shown in Fig 2. The demonstration scenario includes showing walking for a complex series-parallel humanoid robot RH5 which has been recently developed at DFKI-RIC (see Fig. 1).