Neural Feature Extractor for Automotive Small Damage Detection
Small Damage Detection (SDD) is a system that identifies subtle vehicle damage, including dents and scratches. It plays an important role in ensuring safety, accountability, and maintenance in shared mobility environments, such as ride-sharing or car rental services. Identifying when and how a minor dent or scratch occurred is essential for insurance claims, user liability, and cost recovery. Current methods rely on manual inspection or visual comparison using cameras. However, these methods are often slow and subjective. They are unsuitable for real-time use, especially under dynamic conditions or for underbody damage that cameras often miss. Moreover, deploying 360-degree visual systems across fleets adds cost and complexity. To address these gaps, this thesis examines a sensor-based approach. It uses data from a windshield-mounted device equipped with an Inertial Measurement Unit (IMU) and a microphone. The IMU measures acceleration and angular velocity, while the microphone records sound vibrations. While the hardware setup is crucial for data acquisition, the primary focus and contribution of this work are on algorithm development. Deep learning models are applied to detect small damage, as they can learn complex patterns from data. Both supervised (using labelled data) and semi-supervised (using a mix of labelled and unlabelled data) anomaly detection approaches are explored to cope with the scarcity of labelled data.
Feature engineering is crucial in machine learning, as the quality of extracted features directly impacts model performance. Early approaches relied on handcrafted features, which often introduced bias and limited generalisation. Principal Component Analysis (PCA) is one method for automating feature extraction, but it only captures linear dependencies in data. Convolutional layers in supervised networks such as CNNs can model more complex patterns, though they require large labelled datasets. Non-linear autoencoders bridge these approaches by extending PCA to non-linear representations and, when needed, incorporating convolutional layers. Importantly, they can be trained in an unsupervised manner, enabling the learning of compact and expressive feature representations from raw IMU and audio signals without reliance on extensive annotation.
In practice, sensor signals from damage and non-damage events (such as door slams or dents) can appear highly similar, which basic models struggle to distinguish. Addressing this requires data-driven models that can learn non-linear representations and capture complex dependencies between sensor modalities. This motivates the proposed framework in this thesis, which is based on autoencoders and multi-modal sensor fusion. We explore several neural network-based fusion mechanisms, including latent-space pooling, convolution, and attention mechanisms.
The key contribution of this work is an asymmetric multi-modal autoencoder with fusion at the latent representation level, which outperforms mono-modal methods and re-implementations of existing anomaly detection algorithms. This thesis was conducted in an industrial setting, utilising real vehicle field data, thereby ensuring that the evaluation is performed under realistic conditions and enhancing the applicability of the findings. To validate robustness, the system is further evaluated in the robotics domain, using machine sensor data. The results demonstrate generalisability beyond the automotive domain, confirming the effectiveness of the proposed approach.
This thesis demonstrates that cosmetic damage can be detected without the use of cameras, even in dynamic driving scenarios, potentially improving insurance processing, reducing operational costs, and enabling large-scale fleet monitoring.