M-Rock

Human-Machine Interaction Modeling for Continuous Improvement of Robot Behavior

M-Rock is part of the X-Rock developments, enabling users to design personal assistants without any expert knowledge and help domain experts to identify opportunities to improve a system. M-Rock builds on the results of D-Rock and Q-Rock: Modularization and modelling solutions developed in D-Rock enable efficient reuse of components and describe how components can be used in a given context. Q-Rock automatically maps the structural hardware and software complexity of current robotic systems to behaviours. The main goal of M-Rock is to enable the use of user feedback to not only improve behaviours on the software side with respect to the individual requirements of the user, as shown in the Q-Rock outlook, but also to realize a subsequent optimization of the Q-Rock software flow with respect to the hardware selection. M-Rock thus enables an automatic adaptation of a robot to the individual requirements and preferences of the interacting human. For this purpose, explicit feedback (e.g., a rating scale for performance evaluation) is combined with implicit feedback. As a source of implicit human feedback, M-Rock makes use of the EEGs of the users. Using two different rating scenarios, we evaluate the developments in M-Rock to validate that both laymen and domain experts can use them.

Duration: 01.08.2021 till 31.07.2024
Donee: German Research Center for Artificial Intelligence GmbH
Sponsor: Federal Ministry of Education and Research
Grant number: Funded by the Federal Ministry of Education and Research with grant no01IW21002.
Application Field: Assistance- and Rehabilitation Systems
Logistics, Production and Consumer
SAR- & Security Robotics
Underwater Robotics
Space Robotics
Related Projects: D-Rock
Models, methods and tools for the model based software development of robots (06.2015- 05.2018)
Q-Rock
AI-based Qualification of Deliberative Behaviour for a Robotic Construction Kit (08.2018- 07.2021)
Recupera REHA
Full-body exoskeleton for upper body robotic assistance (09.2014- 12.2017)
Related Robots: Quad B12
Quadrupedal Research Platform
RH5
Humanoid robot as an assistance system in a human-optimized environment
RH5 Manus
Humanoid robot as an assistance system in a human-optimized environment
Dual Arm Exoskeleton
Exoskeleton for upper body robotic assistance (Recupera REHA)
Related Software: HyRoDyn
Hybrid Robot Dynamics
Rock
Robot Construction Kit
BOLeRo
Behavior Optimization and Learning for Robots
Phobos
An add-on for Blender allowing editing and exporting of robots for the MARS simulation
MARS
Machina Arte Robotum Simulans
pySPACE
Signal Processing and Classification Environment written in Python

Project details

Graphical User Interface for rating of robot behaviors (Photo: Thomas Röhr, DFKI GmbH)
Current digitalisation developments, AI-based data processing, and powerful hardware lay the groundwork for future embodied AI assistants. These intelligent robots must be versatile, adaptive and flexible to changes in the environment or requirements to become optimised for their purpose for assistance in everyday life and work. Furthermore, they must automatically adapt to perceived new requirements and the changing user's needs.

However, current systems typically do not allow the user to decide how a system should look and behave. This is true for personal assistance, cobots used in production, logistics, or care sectors. Thus, the X-ROCK project series addresses precisely these challenges. X-ROCK enables users to design their personal assistants without any expert knowledge, but it will also help domain experts identify possibilities for improving a system. Modularisation and modelling developed in D-ROCK enables efficient reuse of components and describes how components can be used in a given context. Q-ROCK automatically maps the structural hardware and software complexity of current robotic systems to behaviours.

M-ROCK will directly build on the results of D-ROCK and Q-ROCK. M-ROCK's primary goal is to enable the usage of explicit and implicit user feedback to not only optimise behaviour on the software side to accommodate the user's individual requirements, as shown in Q-ROCK's outlook but also to enable subsequent optimisation of the Q-ROCK software flow, including hardware selection. To this end, we combine explicit feedback (i.e., a rating scale for performance evaluation) with implicit feedback (users' EEG signals).

With the help of two different evaluation scenarios, we will show how the developments in M-ROCK can be used to optimise the software and hardware of a robot by enabling the usage of explicit and implicit human feedback within the Q-ROCK cycle and how it can be used by laymen as well as by domain experts alike.

Videos

RicMonk: A Three-Link Brachiation Robot with Passive Grippers for Energy-Efficient Brachiation

This paper presents the design, analysis, and performance evaluation of RicMonk, a novel three-link brachiation robot equipped with passive hook-shaped grippers. Brachiation, an agile and energy-efficient mode of locomotion observed in primates, has inspired the development of RicMonk to explore versatile locomotion and maneuvers on ladder-like structures. The robot's anatomical resemblance to gibbons and the integration of a tail mechanism for energy injection contribute to its unique capabilities. The paper discusses the use of the Direct Collocation methodology for optimizing trajectories for the robot's dynamic behaviors and stabilization of these trajectories using a Time-varying Linear Quadratic Regulator. With RicMonk we demonstrate bidirectional brachiation, and provide comparative analysis with its predecessor, AcroMonk - a two-link brachiation robot, to demonstrate that the presence of a passive tail helps improve energy efficiency. The system design, controllers, and software implementation are publicly available on GitHub.

M-RoCK+VeryHuman: Whole-Body Control of Series-Parallel Hybrid Robots

The Video illustrates the results of the paper Dennis Mronga, Shivesh Kumar, Frank Kirchner: "Whole-Body Control of Series-Parallel Hybrid Robots", Accepted for Publication: IEEE International Conference on Robotics and Automation (ICRA), 23.5.-27.5.2022, Philadelphia, 2022.

RH5 Manus: Robot Dance Generation based on Music Analysis Driven Trajectory Optimization

Musical dancing is an ubiquitous phenomenon in human society. Providing robots the ability to dance has the potential to make the human robot co-existence more acceptable. Hence, dancing robots have generated a considerable research interest in the recent years. In this paper, we present a novel formalization of robot dancing as planning and control of optimally timed actions based on beat timings and additional features extracted from the music. We showcase the use of this formulation in three different variations: with input of a human expert choreography, imitation of a predefined choreography, and automated generation of a novel choreography. Our method has been validated on four different musical pieces, both in simulation and on a real robot, using the upper-body humanoid robot RH5 Manus.

RH5 Manus: Background of robot dance generation based on music analysis driven trajectory optimization

RH5 Manus: Introduction of a Powerful Humanoid Upper Body Design for Dynamic Movements

Recent studies suggest that a stiff structure along with an optimal mass distribution are key features to perform dynamic movements, and parallel designs provide these characteristics to a robot. This work presents the new upper-body design of the humanoid robot RH5 named RH5 Manus, with series-parallel hybrid design. The new design choices allow us to perform dynamic motions including tasks that involve a payload of 4 kg in each hand,

and fast boxing motions. The parallel kinematics combined with an overall serial chain of the robot provides us with high force production along with a larger range of motion and low peripheral inertia. The robot is equipped with force-torque sensors, stereo camera, laser scanners, high-resolution encoders etc that provide interaction with operators and environment. We generate several diverse dynamic motions using trajectory optimization, and successfully execute them on the robot with accurate trajectory and velocity tracking, while respecting joint rotation, velocity, and torque limits.

RH5 Manus: Humanoid assistance robot for future space missions

The humanoid robot "RH5 Manus" was developed as part of the "TransFIT" project as an assistance robot that can be used in the direct human environment, for example on a future moon station. The aim was to equip the robot with the necessary capabilities to perform complex assembly work autonomously, as well as in cooperation with astronauts and teleoperated. Another focus of the project was on the transfer of the developed technologies to industrial manufacturing and production. The video shows the mechanical assembly and the commissioning of the robot.

RH5: Motion Capture State Feedback for Real-Time Control of a Humanoid Robot

The Video illustrates the results of the paper Mihaela Popescu, Dennis Mronga, Ivan Bergonzani, Shivesh Kumar, Frank Kirchner: "Experimental Investigations into Using Motion Capture State Feedback for Real-Time Control of a Humanoid Robot", Accepted for Publication: MDPI Sensors Journal, Special Issue "Advanced Sensors Technologies Applied in Mobile Robot", 2022.

RH5: Design, Analysis and Control of the Series-Parallel Hybrid RH5 Humanoid Robot

This paper presents a novel series-parallel hybrid humanoid called RH5 which is 2 m tall and weighs only 62.5 kg capable of performing heavy-duty dynamic tasks with 5 kg payloads in each hand. The analysis and control of this humanoid is performed with whole-body trajectory optimization technique based on differential dynamic programming (DDP). Additionally, we present an improved contact stability soft-constrained DDP algorithm which is able to generate physically consistent walking trajectories for the humanoid that can be tracked via a simple PD position control in a physics simulator. Finally, we showcase preliminary experimental results on the RH5 humanoid robot.

Publications

2023

Investigations into Exploiting the Full Capabilities of a Series-Parallel Hybrid Humanoid using Whole Body Trajectory Optimization
Melya Boukheddimi, Rohit Kumar, Shivesh Kumar, Carpentier Justin, Frank Kirchner
In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), (IROS-2023), 01.10.-05.10.2023, Detroit, Michigan, IEEE, Oct/2023.
Classification of error-related potentials evoked during observation of human motion sequences
Su-Kyoung Kim, Julian Liersch, Elsa Andrea Kirchner
In 25th International Conference on Human-Computer Interaction, (HCII-2023), 23.7.-28.7.2023, Copenhagen, Springer, Jul/2023.
Asynchronous classification of error-related potentials in human-robot interaction
Su-Kyoung Kim, Michael Maurus, Mathias Trampler, Marc Tabie, Elsa Andrea Kirchner
In 25th International Conference on Human-Computer Interaction, (HCII-2023), 23.7.-28.7.2023, Copenhagen, Springer, Jul/2023.
AcroMonk: A Minimalist Underactuated Brachiating Robot
Mahdi Javadi, Daniel Harnack, Paula Stocco, Shivesh Kumar, Shubham Vyas, Daniel Pizzutilo, Frank Kirchner
In IEEE Robotics and Automation Letters, IEEE, volume 8, pages 1-8, Jun/2023.
LQR for Free-Floating Robots: Theory and Experiments
Shubham Vyas, Lasse Shala, Anton Bredenbeck
In 17th Symposium on Advanced Space Technologies in Robotics and Automation, (ASTRA-2023), 18.10.-20.10.2023, Leiden, ESA, 2023. ESA.
Linear Model Predictive Control for a planar free-floating platform: A comparison of binary input constraint formulations
Franek Stark, Shubham Vyas, Georg Schildbach, Frank Kirchner
In 17th Symposium on Advanced Space Technologies in Robotics and Automation, (ASTRA-2023), 18.10.-20.10.2023, Leiden, ESA, 2023. ESA.
Fast Dynamic Walking with RH5 Humanoid Robot
Ivan Bergonzani, Mihaela Popescu, Shivesh Kumar, Frank Kirchner
In 2023 IEEE-RAS 22nd International Conference on Humanoid Robots (Humanoids), (Humanoids), 12.12.-14.12.2023, Austin, TX, IEEE, 2023. ISBN: 979-8-3503-0327-8.

2022

Quantifying the Effect of Feedback Frequency in Interactive Reinforcement Learning for Robotic Tasks
Daniel Harnack, Julie Pivin-Bachler, Nicolas Navarro-Guerrero
In Neural Computing and Applications, Springer, volume n.n., pages 0-0, Dec/2022.
Experimental Investigations into Using Motion Capture State Feedback for Real-Time Control of a Humanoid Robot
Mihaela Popescu, Dennis Mronga, Ivan Bergonzani, Shivesh Kumar, Frank Kirchner
In Sensors - Open Access Journal, Multidisciplinary Digital Publishing Institute (MDPI), volume 22, number 24, pages 1-12, Dec/2022.
Co-Optimization of Acrobot Design and Controller for Increased Certifiable Stability
Lasse Jenning Shala, Felix Wiebe, Shivesh Kumar, Mahdi Javadi, Frank Kirchner
In Proceedings of the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2022), (IROS-2022), 23.10.-27.10.2022, Kyoto, IEEE, Nov/2022.
Robot Dance Generation with Music Based Trajectory Optimization
Melya Boukheddimi, Daniel Harnack, Shivesh Kumar, Rohit Kumar, Shubham Vyas, Octavio Arriaga, Frank Kirchner
In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2022, (IROS-2022), IEEE, Nov/2022.
Design optimization of a parallel manipulator for otological surgery
Durgesh Haribhau Salunkhe, Guillaume Michel, Elise Olivier, Shivesh Kumar, Marcello Sanguineti, Damien Chablat
In 2022 Workshop: "New frontiers of parallel robotics" (second edition), (ICRA-2022), 23.5.-27.5.2022, Philadelphia, PA, IEEE, May/2022.
An efficient combined local and global search strategy for optimization of parallel kinematic mechanisms with joint limits and collision constraints
Durgesh Haribhau Salunkhe, Guillaume Michel, Shivesh Kumar, Marcello Sanguineti, Damien Chablat
In Mechanism and Machine Theory, Elsevier, volume 173, pages 1-32, Apr/2022.
Finding Optimal Placement of the Almost Spherical Parallel Mechanism in the Recupera-Reha Lower Extremity Exoskeleton for Enhanced Workspace
Ibrahim Tijjani
Editors: Andreas Müller, Mathias Brandstötter
In Advances in Service and Industrial Robotics, (RAAD-2022), 08.6.-10.6.2022, Klagenfurt am Wörthersee, Springer International Publishing, series Mechanisms and Machine Science, volume 120, pages 536-544, Apr/2022. ISBN: 978-3-031-04870-8.
Bidirectional and Coadaptive Robotic Exoskeletons for Neuromotor Rehabilitation and Assisted Daily Living: a Review
Elsa Andrea Kirchner, Judith Bütefür
In Current Robotics Reports, Springer Nature, volume N./A., pages 2662-4087, Apr/2022.
A Survey on Design and Control of Lower Extremity Exoskeletons for Bipedal Walking
Ibrahim Tijjani, Shivesh Kumar, Melya Boukheddimi
In Applied Sciences, MDPI, volume 12, number 5, pages 1-31, Feb/2022.
Whole-Body Control of Series-Parallel Hybrid Robots
Dennis Mronga, Shivesh Kumar, Frank Kirchner
In IEEE International Conference on Robotics and Automation (ICRA), (ICRA-2022), 23.5.-27.5.2022, Philadelphia, IEEE, pages 228-234, 2022.
Introducing RH5 Manus: A Powerful Humanoid Upper Body Design for Dynamic Movements
Melya Boukheddimi, Shivesh Kumar, Heiner Peters, Dennis Mronga, Rohan Budhiraja, Frank Kirchner
In IEEE International Conference on Robotics and Automation (ICRA), (ICRA-2022), 23.5.-27.5.2022, Philadelphia, IEEE, pages 01-07, 2022. ISBN: 978-1-7281-9681-7.
RealAIGym: Education and Research Platform for Studying Athletic Intelligence
Felix Wiebe, Shubham Vyas, Lasse Jenning Shala, Shivesh Kumar, Frank Kirchner
Editors: Brian Plancher, Dylan Shell, Kris Hauser, Shuran Song, Katja Mombaur
In Proceedings of the Robotics: Science and System Workshop Mind the Gap: Opportunities and Challenges in the Transition Between Research and Industry, 1.7.-1.7.2022, New York, New York, Robotics Science and Systems, Online Proceedings, 2022. RSS Foundation.
Post-Capture Detumble Trajectory Stabilization for Robotic Active Debris Removal
Shubham Vyas, Lasse Maywald, Shivesh Kumar, Marko Jankovic, Andreas Mueller, Frank Kirchner
In Advances in Space Research, Elsevier Ltd., volume 1, pages 1-18, 2022.

Back to the list of projects
© DFKI GmbH
last updated 04.01.2024
to top