Learning from Causal Physical World Models

The project aims at reducing the sample and model complexity of robot learning algorithms. For that, known physical equations are merged with machine learning models in order to adapt faster to unknown environments while quantifying uncertainty. Causal learning will be applied to this world model to generate exploratory behavior and to accelerate the learning processes. Once a world model has been learned, robot behaviors will be optimized using optimal control or reinforcement learning.

Duration: 01.07.2022 till 31.12.2024
Donee: German Research Center for Artificial Intelligence GmbH & University of Bremen
Sponsor: Federal Ministry for Economic Affairs and Climate Action
Grant number: DFKI: 50RA2126A Uni: 50RA2126A
Application Field: Space Robotics
Related Robots: UR5+10 Dual Arm System

Project details

Physics simulations have been proven to be beneficial in a lot of use-cases where real-world learning is infeasible or impractical. In order to minimize risks, costs, and above all, the time required for real-world learning, the PhysWM project leverages physics simulation engines to learn new behaviors for robotic scenarios. There are, however, certain problems associated with simulation. For instance, the physics implementation in a simulation engine is not sufficiently accurate to efficiently approximate physical processes. In order to ensure that the behaviors learned in simulation remain functional when transferred to reality, critical aspects of the simulation are often randomized (domain/dynamics randomization). This requires additional simulation time and expert knowledge regarding suitable parameters. Additionally, the simulation may not account for all real-world scenarios and unknown factors that can affect the behavior of the robotic system, leading to potential issues when deployed in the real world.

In PhysWM, we develop algorithms and software components that combine modern, differentiable physics engines with learning methods that enable robotic systems to develop a physically plausible world model largely autonomously. We propose a hybrid simulator consisting of differentiable physics and rendering modules. This hybrid model of universal prior physics knowledge and neural networks allows gradient-based optimization techniques to be applied and a direct estimation of model uncertainty can be performed. Both are success factors for efficient adaptation to new environments and for precise explainability of learned models and behavior, which are key challenges for the autonomy of robotic systems.

In PhysWM, a curriculum learning approach is being implemented to optimize the effort for the improvement of behavior learning. Curriculum learning is applied to both task and model complexity to guide exploration behaviors and accelerate the learning process. This approach starts by learning the behavior of robots in simple world models and gradually increasing the complexity as needed. Moreover, Bayesian inference allows to estimate the uncertainty over physical parameters used in our model. This curriculum-based approach is expected to improve the overall efficiency and accuracy of the robotic system's behavior learning.



PhysWM: Physical World Models for Robot Learning
Marc Otto, Octavio Arriaga, Chandandeep Singh, Jichen Guo, Frank Kirchner
In NeSy 2023: 17th International Workshop on Neural-Symbolic Learning and Reasoning, (NeSy-2023), 3.7.-5.7.2023, Certosa di Pontignano, Siena, CEUR Workshop Proceedings, Jul/2023.

Back to the list of projects
last updated 04.01.2024