Sonar2Depth: Acoustic-Based 3D Reconstruction Using cGANs

Abstract :

This work proposes the use of conditional Generative Adversarial Networks (cGANs) for acoustic-based 3D reconstruction. Acoustics being the most reliable sensor modality in underwater domains is accompanied with the loss of elevation angle in its images. The challenge of recovering the missing dimension in acoustic images have pushed researchers to try various methods and approaches over the past years. cGANs being an image-to-image translation method makes it possible to learn a desired style, and transforms the data from one modality to another. This was applied here as a way of transforming an acoustic image into another form which contains the elevation characteristics, such as depth images. Depth images are hard to acquire underwater, thus data was generated synthetically and used for training and testing the deep learning model. As a way of performance enhancement, real data was collected for training a Cycle-GAN network in the aim of transferring the realistic style into the synthetically generated images. Simulation experiments were conducted to evaluate the system and find out the best experimental setup, which was then used to carry out the real experiment. The system performed dense 3D reconstruction of the scanned object and proved to be applicable in real environments.

Keywords :

3D reconstruction, imaging sonar, marine robotics, deep-learning, GANs

Links:

https://ieeexplore.ieee.org/abstract/document/10342251