PhysWM

Physics simulations have been proven to be beneficial in a lot of use-cases where real-world learning is infeasible or impractical. In order to minimize risks, costs, and above all, the time required for real-world learning, the PhysWM project leverages physics simulation engines to learn new behaviors for robotic scenarios. There are, however, certain problems associated with simulation. For instance, the physics implementation in a simulation engine is not sufficiently accurate to efficiently approximate physical processes. In order to ensure that the behaviors learned in simulation remain functional when transferred to reality, critical aspects of the simulation are often randomized (domain/dynamics randomization). This requires additional simulation time and expert knowledge regarding suitable parameters. Additionally, the simulation may not account for all real-world scenarios and unknown factors that can affect the behavior of the robotic system, leading to potential issues when deployed in the real world.

In PhysWM, we develop algorithms and software components that combine modern, differentiable physics engines with learning methods that enable robotic systems to develop a physically plausible world model largely autonomously. We propose a hybrid simulator consisting of differentiable physics and rendering modules. This hybrid model of universal prior physics knowledge and neural networks allows gradient-based optimization techniques to be applied and a direct estimation of model uncertainty can be performed. Both are success factors for efficient adaptation to new environments and for precise explainability of learned models and behavior, which are key challenges for the autonomy of robotic systems.

In PhysWM, a curriculum learning approach is being implemented to optimize the effort for the improvement of behavior learning. Curriculum learning is applied to both task and model complexity to guide exploration behaviors and accelerate the learning process. This approach starts by learning the behavior of robots in simple world models and gradually increasing the complexity as needed. Moreover, Bayesian inference allows to estimate the uncertainty over physical parameters used in our model. This curriculum-based approach is expected to improve the overall efficiency and accuracy of the robotic system's behavior learning.