Detection of action deviations through learning with limited computing resources

The HaLer project aims to develop solutions for flexible and adaptive human-machine interaction (MMI). The developed methods should allow a system to react and adapt flexibly to human behavior even in the case of minor changes in the situation. For this purpose, deviations in human behavior based on behavioral data and EEG data are to be detected. Furthermore, the combined use of behavioral data and EEG data should make it possible to predict upcoming sequences of action.

Duration: 01.08.2020 till 31.12.2022
Donee: German Research Center for Artificial Intelligence GmbH & University of Bremen
Sponsor: Federal Ministry for Economic Affairs and Climate Action
German Aerospace Center e.V.
Grant number: DFKI: 50RA2024 Uni: 50RA2023
Application Field: Space Robotics
Related Projects: BesMan
Behaviors for Mobile Manipulation (05.2012- 07.2016)
Intelligent Man-Machine Interface - Adaptive Brain-reading for assistive robotics (05.2010- 04.2015)
Flexible Interaction for infrastructures establishment by means of teleoperation and direct collaboration; transfer into industry 4.0 (07.2017- 12.2021)
AI-based Qualification of Deliberative Behaviour for a Robotic Construction Kit (08.2018- 07.2021)
Adaptive software framework for context-sensitive, intuitive man-machine-interaction (06.2020- 12.2023)
Exploring the Potential of Pervasive Embedded Brain Reading in Human Robot Collaborations (06.2020- 05.2024)
Related Robots: COMPI
Compliant Robot Arm
Related Software: pySPACE
Signal Processing and Classification Environment written in Python
Reconfigurable Signal Processing and Classification Environment
Behavior Optimization and Learning for Robots

Project details

In the HaLeR project, solutions for a flexible and adaptive MMI are to be developed. The developed methods should allow a system to react and adapt flexibly to human behavior when situations change. To this end, the University of Bremen will develop an approach that automatically recognizes the connections between individual behavioral segments and adapts them to new, similar situations so that upcoming behavioral segments can be predicted in a situationally appropriate manner. Thus, deviations in human behavior are detected. DFKI Bremen develops scenarios to test whether deviations in human behavior are detected when situations change and whether the behavior of the system within a man-machine collaboration is adapted accordingly. In this way, the flexibility of the MMI can be evaluated, e.g. it is evaluated whether and how the system successfully supports the human being despite changed movement (e.g. changed sequence of behavioral segments).

In order to detect deviations in the human's actions, both the human's movements by means of motion tracking and intrinsic motivations and evaluations of current situations are analyzed using EEG data. The data will be analyzed using learning methods developed by the Robotics Group of the University of Bremen, which has great expertise in the development of adaptive and robust learning architectures and learning methods. The focus is on methods that do not require large computing capacities. Thus, the methods can be applied in scenarios with limited resources, such as space missions. The focus of DFKI's work is on the development of test environments and the evaluation of methods developed by the robotics group at the University of Bremen.



Classification of error-related potentials evoked during observation of human motion sequences
Su-Kyoung Kim, Julian Liersch, Elsa Andrea Kirchner
In 25th International Conference on Human-Computer Interaction, (HCII-2023), 23.7.-28.7.2023, Copenhagen, Springer, Jul/2023.


The influence of labeling techniques in classifying human manipulation movement of different speed.
Sadique Adnan Siddiqui, Lisa Gutzeit, Frank Kirchner
In International Conference on Pattern Recognition Applications and Methods, (ICPRAM-2022), 03.2.-05.2.2022, n.n., pages 338-345, Feb/2022. ISBN: 978-989-758-549-4.


A Comparison of Few-Shot Classification of Human Movement Trajectories
Lisa Gutzeit
In Proceedings of the 10th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, (ICPRAM-2021), 04.2.-06.2.2021, SciTePress, pages 243-250, Feb/2021. ISBN: 978-989-758-486-2.

Back to the list of projects
last updated 04.01.2024