Haptic sensors span a broad range of technologies. The main focus of the sensors is to increase the recognition accuracy of both textures and the location of contact points. However, these sensors are mechanically fragile and mounted externally to robotic systems to increase accuracy, limiting the use of those sensors to applications that are kind to the sensors. For use in harsh applications or complementary to those existing sensors, this project aims to develop a machine learning-oriented solution capable of using body-borne vibrations to classify objects' texture and location of haptic interaction. This strategy allows mounting the sensors inside the robot, protected from external perturbance. Although this technology is not as accurate as other technologies, it promises to enable a degree of haptic perception anywhere the robot's outer shell (and electronics) can withstand.
The technology has been validated in applications of multimodal object recognition, e.g., by Bonner et al. 2021 and Toprak et al. 2018. Some of the following steps include the development of the algorithms to perform localization of multiple points of contact between the robot and external objects.
Systematic collect sound and vibration data from a robotic hand
Determine ideal placement of sensors
Perform sound-source localization of the source of the vibration or sound within the robotic hand
Machine Learning, Python
A Electronics or Mechatronics background
Navarro-Guerrero, N., Toprak, S., Josifovski, J., & Jamone, L. (2022). Visuo-Haptic Object Perception for Robots: An Overview. Autonomous Robots. doi.org/10.48550/arXiv.2203.11544
Bonner, L. E. R., Buhl, D. D., Kristensen, K., & Navarro-Guerrero, N. (2021). AU Dataset for Visuo-Haptic Object Recognition for Robots. figshare. doi.org/10.6084/m9.figshare.14222486
Toprak, S., Navarro-Guerrero, N., & Wermter, S. (2018). Evaluating Integration Strategies for Visuo-Haptic Object Recognition. Cognitive Computation, 10(3), 408–425. doi.org/10.1007/s12559-017-9536-7
Deutsches Forschungszentrum für Künstliche Intelligenz GmbH
Robotics Innovation Center
28359 Bremen, Germany
Phone: +49 421 17845 4119