Multimodal object recognition is still an emerging and active field of research. Haptic sensors span a broad range of technologies. The main focus of the sensors is to increase the recognition accuracy of both textures and the location of contact points. However, these sensors are mechanically fragile and mounted externally to robotic systems to increase accuracy, limiting the use of those sensors to applications that are kind to the sensors. For use in harsh applications or complementary to those existing sensors, this project aims to develop a machine learning oriented solution capable of using body-borne vibrations to classify objects' texture and location of haptic interaction. This strategy allows mounting the sensors inside the robot, protected from external perturbance. Although this technology is not as accurate as other technologies, it promises to enable a degree of haptic perception anywhere the robot outer shell (and electronics) can withstand.
The technology has been validated in applications of multimodal object recognition, e.g., by Bonner et al. 2021 and Toprak et al. 2018. Some of the following steps include 1) the development of "robotic fingerprints" that maximize the body-borne vibrations, 2) and later developing the algorithms to perform localization of multiple points of contact between the robot and external objects.
≫ Development of 3D printed robotic fingerprints for robotic grippers.
≫ Systematic optimization of fingerprint's texture and material for different applications, underwater manipulation, humanoid robots, etc.
≫ Creation of a public haptic dataset
≫ Rapid prototyping, including CAD and 3D Printing
≫ Navarro-Guerrero, N., Toprak, S., Josifovski, J., & Jamone, L. (2022). Visuo-Haptic Object Perception for Robots: An Overview. Autonomous Robots. doi.org/10.48550/arXiv.2203.11544
≫ Bonner, L. E. R., Buhl, D. D., Kristensen, K., & Navarro-Guerrero, N. (2021). AU Dataset for Visuo-Haptic Object Recognition for Robots. figshare. doi.org/10.6084/m9.figshare.14222486
≫ Toprak, S., Navarro-Guerrero, N., & Wermter, S. (2018). Evaluating Integration Strategies for Visuo-Haptic Object Recognition. Cognitive Computation, 10(3), 408–425. doi.org/10.1007/s12559-017-9536-7
Deutsches Forschungszentrum für Künstliche Intelligenz GmbH
Robotics Innovation Center
28359 Bremen, Germany
Phone: +49 421 17845 4119