Self-supervised Learning for Sonar Images: Enhancing Multimodal Perception for Underwater Applications

Neural networks have shown promising results in sonar perception tasks such as object recognition [1], image patch matching [2] and image classification [3]. In the context of autonomous underwater vehicles, it is crucial to develop robust, accurate and efficient models to overcome the challenges of underwater perception.

In this work, we report progress on a comparative evaluation of self-supervised learning (SSL) [6][7] and supervised learning (SL) algorithms as pre-training methods for sonar images. As a first step, we produce pre-trained neural networks on the Marine Debris Watertank Dataset [4] via a SSL method that classifies image rotations [5] and a traditional SL approach to classify the actual image labels. In both cases, we trained a Resnet20, SqueezeNet, Mobilenet, DenseNet121 and MiniXception on images of size 96x96. Thereafter, we evaluate the quality of the learned features by using transfer learning for low-shot classification on a target dataset called Marine Debris Turntable [3].

The results presented in this poster indicate that the SSL pre-trained models have a similar classification performance compared to the SL counterpart across all the neural network models. These results indicate that SSL pre-training are a promising substitute for SL methods without compromising object classification and no need of human manual annotations.

Finally, we report on the creation of a new underwater dataset that contains paired camera and sonar images for different underwater objects (panels, cement pipes, ladders, ramps). This dataset, called Gemini Sonar Dataset, will allow us to perform further classification, image translation and object detection tasks using SSL approaches.

----------------------------------------------------
[1] M. Valdenegro-Toro. Object recognition in forward-looking sonar images with convolutional neural networks. In OCEANS 2016 MTS/IEEE Monterey, pages 1–6, 2016. doi: 10.1109/OCEANS.2016.7761140.

[2] M. Valdenegro-Toro. Improving sonar image patch matching via deep learning. In 2017 European Conference on Mobile Robots (ECMR), pages 1–6, 2017. doi: 10.1109/ECMR.2017.8098701.

[3] Matias Valdenegro-Toro, Alan Preciado-Grijalva, Bilal Wehbe. Pre-trained Models for Sonar Images. Global OCEANS, 2021.

[4] Deepak Singh, Matias Valdenegro-Toro. The Marine Debris Dataset for Forward-Looking Sonar Semantic Segmentation. ArXiv:2108.06800, 2021.

[5] Spyros Gidaris, Praveer Singh, Nikos Komodakis. Unsupervised representation learning by predicting image rotations, 2018.

[6] Longlong Jing, Yingli Tian. Self-supervised visual feature learning with deep neural networks: A survey. CoRR, abs/1902.06162, 2019. URL
arxiv.org/abs/1902.06162.

[7] Ashish Jaiswal, Ashwin Ramesh Babu, Mohammad Zaki Zadeh, Debapriya, Banerjee, Fillia Makedon. A survey on contrastive self-supervised learning, 2021.