Hybrid and intelligent human-robot collaboration – Hybrid teams in versatile cyber-physical production environments

The joint research project „Hybrid and intelligent human-robot collaboration: hybrid teams in versatile cyber-physical production environments – Hybr-iT” investigates and tests in an industrial context the components that are essential for the planning and optimization of hybrid teams, for their integration in existing IT and production infrastructures as well as for their control in a production plant. Hybrid teams are composed in this project of several humans (m>=2) and robots (n>=2) as well as software-based assistant systems. A key element is that all the developments of the project need to be seamless and smoothly integrated in existing industrial infrastructures. This requires that all developments are modular and hardware-independent.

Duration: 01.11.2016 till 31.10.2019
Donee: German Research Center for Artificial Intelligence GmbH
Sponsor: Federal Ministry of Education and Research
Grant number: Funded by BMBF (German Federal Ministry of Education and Research) Grant Nr: 01IS16026A
Website: http://hybr-it-projekt.de/
Partner: Airbus Operations GmbH, KUKA Roboter GmbH, VOLKSWAGEN AKTIENGESELLSCHAFT, Broetje-Automation GmbH, EngRoTec GmbH & Co. KG, The Captury GmbH, Federal Institute for Occupational Safety and Health (BAuA)
Application Field: Logistics, Production and Consumer
Related Projects: HySociaTea
Hybrid Social Teams for Long-Term Collaboration in Cyber-Physical Environments (09.2014- 08.2016)
Intelligent Human-Robot Collaboration (03.2015- 06.2016)
Behaviors for Mobile Manipulation (05.2012- 07.2016)
Models, methods and tools for the model based software development of robots (06.2015- 05.2018)
Internal Logistics with Automated Autonomous Delivery and Replenishment (01.2016- 12.2016)
Related Robots: iMRK
iMRK Dual-Arm Robot (Photo: Annemarie Popp, DFKI GmbH)
Related Software: ARC-OPT
Adaptive Robot Control using Optimization
Federal Ministry of Education and Research

Project details

Figure 1. Schematic diagram of the architecture in Hybr-iT (Photo: Renato Orsini, DFKI GmbH)
The goal of the project is the development and testing of hybrid teams in an industrial environment. Those teams are composed of humans, robots and software-based assistant systems as well as intelligent virtual environments and are, from an IT perspective, highly-distributed systems which include very different IT and cyber-physical subsystems: robot controllers, safety, logistics and tracking systems, databases, simulation and visualization environments, etc. In order to achieve the ambitioned collaborative application, those subsystems need to be interconnected with highest performance.

One essential component and scientific challenge for the deployment of hybrid teams is a comprehensive resource-oriented architecture (ROA) for the interconnection of heterogeneous cyber-physical systems and IT environments. This architecture allows a dynamic connection between all required subsystems (IT environments, cyber-physical systems and multimodal interaction and control hardware) as well as their individual further development with the help of widely-standardized interfaces and well-defined data models. Besides, for the resource-oriented architecture (Figure 1), a safe and real-time communication layer is indispensable.

Furthermore, a software architecture for the robot itself is required which seamlessly interacts with the ROA and establishes the information for the control of robots in hybrid teams. This architecture abstracts from the used hardware and allows a robot-independent deployment of the developed solutions. Through the interplay of ROA and robot architecture, IT environments and robots can be interconnected regardless of manufacturer.

In order to keep IT and cyber-physical subsystems running synchronous and consistently, a real-time, adaptive, and semantic data and environment model is also required, which receives all the required information from the subsystems and can update that of the ROA and of the robot architecture. A further prerequisite for a close cooperation between workers and robots is a robust, precise, and extensive real-time worker recognition which works under very different production conditions and also with changing workforce. In order to minimize setting-up times and avoid disturbing markers, marker-less tracking procedures need to be further developed so that they can also work precisely under industrial conditions.

To achieve a direct human-robot interaction, i.e. to communicate between humans, robots and assistant systems, multimodal dialogue systems are required, which use both already-existing input and output channels (e.g. switches, monitors, acoustic signals, etc.) and also new possibilities such as speech interaction, gesture recognition and haptic feedback. Those dialogue systems will access both the robot architecture and the ROA.

Together with the industrial partners, DFKI will implement two evaluation scenarios for the demonstration of hybrid teams in the automotive and aerospace assembly. Several DFKI research groups work together in the project Hybr-iT. The Robotics Innovation Center has its focus on the development of safe and intuitive human-robot collaboration and the therefor required robot skills.


Hybr-iT: Knowledge-Based Object Manipulation - First Vertical Software Integration

The video shows the first vertical integration of software components for knowledge-based object manipulation within the Hybr-iT project. These components include the task planner CHIMP, a semantically enriched environment representation, the event-based high-level robot controller and monitor Xec and the real-time motion controller of the robot. The integration of these components is demonstrated by the means of a context-based manipulation task on an industrial dual-arm robot: Picking and screwing a gearbox mount. The task is executed in two different variants: Once with the support of a human worker and once autonomously by the robot. Thereby, the robot behavior is adapted online according to the task context.

Mobipick: System demonstration

Hybr-iT: Knowledge-Based Robot Skills for Collaborative Manufacturing in Hybrid Teams

This video shows human-robot collaboration for assembly of a gearbox mount in a realistic replica of a production line of Volkswagen AG. Knowledge-based robot skills enable autonomous operation of a mobile dual arm robot side-by-side of a worker.



Learning Task Constraints for Whole-Body Control of Robotic Systems
Dennis Mronga
May/2022. Universität Bremen.


A Constraint-Based Approach for Human-Robot Collision Avoidance
Dennis Mronga, Tobias Knobloch, José de Gea Fernández, Frank Kirchner
In Advanced Robotics, Taylor & Francis Online, volume 0, pages 1-17, 2020.


Towards Contextual Robots for Collaborative Manufacturing
Jose de Gea Fernandez, Dennis Mronga, Martin Günther, Sebastian Stock, Nils Niemann, Hendrik Wiese, Rohit Menon, Elsa Andrea Kirchner, Stefan Stiene
Editors: Valerio Ortenzi, Marco Controzzi
In Poster at the Workshop "Human-Robot Cooperation and Collaboration in Manipulation: Advancements and Challenges" at 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2018)., 01.10.-05.10.2018, Madrid, o.A., 2018.
Hybr-iT Project: Initial Steps Towards Contextual Robotic Manipulation for Human-Robot Teams in Industrial Environments
José de Gea Fernández, Nils Niemann, Sebastian Stock, Martin Günther, Dennis Mronga, Hendrik Wiese, Rohit Menon, Elsa Andrea Kirchner, Stefan Stiene
Poster at the 21st International Conference of the Catalan Association for Artificial Intelligence (CCIA 2018), 2018.


Multimodal Sensor-Based Whole-Body Control for Human-Robot Collaboration in Industrial Settings
José de Gea Fernández, Dennis Mronga, Martin Günther, Tobias Knobloch, Malte Wirkus, Martin Schröer, Mathias Trampler, Stefan Stiene, Elsa Andrea Kirchner, Vinzenz Bargsten, Timo Bänziger, Johannes Teiwes, Thomas Krüger, Frank Kirchner
In Robotics and Autonomous Systems, Elsevier, volume 94, pages 102-119, Aug/2017.

Back to the list of projects
last updated 02.03.2023
to top