Online Adaptation of a Man-Machine Interface with Respect to Task Engagement and Task Load

Zusammenfassung (Abstract) :

For remote control of robots in places not easily accessible by humans, advanced man-­‐ machine interfaces (MMIs) are developed. Such MMIs immerse an operator into the scenario to provide the feeling of being ”on-­‐site”. In this work, the improvement of user support and efficiency of interaction by means of embedded Brain Reading (eBR) was investigated within a robotic application scenario. In our approach, online adaptation of an MMI was controlled with respect to the level of task engagement inferred from brain activity that is naturally evoked during interaction. An advanced MMI was developed that makes use of a virtual control environment such as a Cave Automatic Virtual Environment (CAVE). To meet the objective, the developed MMI was adapted based on the changes in task engagement of the user measured by brain activity, i.e., the electroencephalogram (EEG). We used EEG activity related to the P300 event-­‐related potential (ERP) to derive the user’s current task engagement. P300 related activity is often used to derive the processing capability of the brain. However, our approach avoids the usage of a secondary task to measure task load. We assume that the amplitude of a single-­‐trial P300 evoked by a new task message is reduced in case of the user still being involved in a former task, since attentional, perceptional and other cognitive resources are shared between the former and the new task. Moreover, we expected that the frequent occurrence of such situations lead to a reduction of the averaged P300 peak amplitude. To prove this, we artificially modulated the level of task engagement by using two different inter-­‐stimulus intervals (ISIs): long ISIs and short ISIs. Our assumption was that the change in task engagement, which was provoked by the modulation of ISI, correlates with the averaged P300 amplitude as well as its single-­‐ trial detectability. Based on our assumption, we adapted the developed MMI by changing the ISI depending on task engagement of the user, which was measured by online single-­‐ trial classification (e.g., if P300 related activity was detected or not, which presumably depends on P300 amplitude). By means of this approach, the next ISI was extended when task engagement was enhanced (i.e., P300 related activity was not detected), whereas the next ISI was reduced when less task engagement was observed (i.e., P300 related activity was detected). We performed an experiment with six subjects. EEG and behavioral data was recorded during low task frequency (long ISI), high task frequency (short ISI) and online adapted task frequency (adapted ISI) conditions. Expected changes in the average P300 peak amplitude could be confirmed. As a main result we showed a significant reduction of runtime needed to perform the interaction tasks under the adapted ISI condition. Subjects further reported that the adapted ISI provided just the right task frequency indicating a positive effect on the interaction. Moreover, the analyzed behavioral data did not show significant differences between conditions indicating that brain activity might, in our case (i.e., a complex multitasking scenario), be a better predictor for user states than behavioral data.

Files:

2015_BWMMS_Fokus_Trac_Talk_Abstrakt_EAKirchner.pdf