Comparison of VTOL UAV Battery Level for Propeller Faulty Classification Model

—T he degradation of batteries in UAVs may result in various problems, such as connectivity troubles, flight delays, and unexpected accidents. Flight safety and reliability are affected by propeller efficiency and performance. This study explores an acoustic-based method to classify propeller faulty conditions in Vertical Take-Off and Landing Unmanned Aerial Vehicles (VTOL UAV). The main objective is to emphasize the difference between classifier models developed using different battery-level flight data. The sound generated by VTOL UAV provides valuable information about the flight performance, essential for effectively monitoring flying conditions and identifying potential faults. This study uses three classification algorithms-Medium Tree (MT), Linear Support Vector Machine (LSVM), and Linear Discriminant (LD), to classify propeller failures of VTOL UAVs. Datasets are collected from three simulated propeller faulty conditions using a wireless microphone connected to a smartphone in an indoor lab environment with a soundproofing mechanism. The Mel Frequency Cepstral Coefficients technique is implemented in MATLAB (R2020a) to extract valuable features from the recorded sound signals. Extracted features from high and low-battery flights are utilized to develop classification models. Classifiers' performance is analyzed to compare the difference between selected models developed using high and low-battery flight data. The accuracy was measured with other samples to test the robustness of classification models. LSVM and MT classification models developed using high-battery flight data produce better accuracy than low-battery flight data in the training and testing phases. LD classification model developed using high-battery flight data produces better accuracy than low-battery flight data in the testing phase only. These results show that battery degradation can affect the performance of the VTOL UAV faulty classification algorithm.


I. INTRODUCTION
VTOL UAV (Vertical Take-Off and Landing Unmanned Aerial Vehicle) can take off, fly, hover, and land vertically.Due to the many benefits compared to manned vehicles, VTOL UAVs are in high demand in the marketplace [1].Over the past decade, UAVs have been employed in various applications, including crop monitoring, surveillance and monitoring, transportation, building systems, delivery systems, and inspection [2].The military and civilian sectors increasingly use unmanned aerial vehicles for various tasks, which might have a terrible effect if any malfunctions occur during flight.
General flight failures in UAVs could result from propeller, eccentric, and bearing malfunctions, significantly affecting flight performance [3], [4].Recent studies have also shown actuator malfunctions as a significant source of UAV failure in flight, both in military and commercial UAVs [5].Most of the studies stated that the faults of the VTOL UAV result from malfunctions in its fundamental parts.For these reasons, condition monitoring and fault identification are essential issues in UAVs.
Various experiments have been conducted to study specific failures that affect VTOL UAV flight performance using different sensors.Extensive time series and frequency domain analysis of collected data from multiple sensors could extract hidden information for failure identification.For example, Ray et al. [6] investigate the inter-turn short circuit faults in the motor winding of single-phase UAV systems using multiresolution analysis based on statistical parameter estimation for monitoring.Altinors et al. [4] proved that multiple faults from the UAV main component (see Fig. 1) could be detected and identified.Researchers around the globe have conducted different studies to investigate various faulty detection measures focusing on motors and actuators with similar aims to avoid flight crashes and retain stability in failure conditions.Cheng et al. [7] address the problem of UAV faults and diagnose UAVs' health status by measuring UAV motor vibration.The motor's vibration was measured by altering the propeller condition and asymmetrical motor mount pattern.A study by Huimin et al. [8] developed an anomaly detection system that can prevent the motor of a drone from operating at abnormal temperatures to reduce the frequency of UAV crashes.Benini et al. [9] proposed a diagnostic algorithm for actuator fault detection in VTOL UAVs.Park et al. [10] proposed multivariate statistical analysis techniques on the inertial measurement unit (IMU) and the motor input measurements to isolate an actuator fault in a quadrotor.A combined investigation involving the motor and propeller was done by Lee et al. [11].They developed an overall fault diagnostic technique for the UAV by considering the broken propeller for malfunctioning the UAV motor.
The flexibility of the UAV propeller plays a vital role in the dynamics of flight conditions.Propeller cracks and bent are the most common faults detected on VTOL UAVs in actual operating environments.Cahabug et al. [12] proposed a failure detection system for a UAV that detects propeller failures to reduce the risk of crashes.Zhang et al. [13] developed a simulation model to achieve high accuracy in detecting propeller faults in flight.
An exhaustive study is needed to diagnose how propeller efficiency and performance affect flight safety and reliability.Ghalamchi et al. [14] proposed an estimator for detecting and diagnosing propeller degradation on a multicopter aerial robot.Palanisamy et al. [15] implemented an extended Kalman filter-based parameter estimation algorithm to identify changes in the propeller aerodynamic efficiency.The author focused on propeller blade performance and damage detection in electric UAVs.Ahsun et al. [16] present a recursive algorithm for estimating a propeller engine's thrust and power coefficient.Nemati et al. [17] derived a dynamic model of a tilting-rotor quadcopter with one propeller failure and designed a controller to achieve hovering and navigation capability.
The degradation of batteries in UAVs may result in various problems, such as connectivity troubles, flight delays, and unexpected accidents.Therefore, battery faults or depletion could make it more challenging for UAVs to operate reliably.In the literature, there appear to be limited studies on the battery performance of UAVs.Mohsan et al. [18] stated that charging UAVs is one of the most time-consuming and complex activities.Due to their short battery life, UAVs' mission duration and range are limited.Tseng et al. [19] conducted research to identify how a UAV's power consumption is affected by movement (including hovering, vertical, and horizontal movement), payload, and wind.
Detecting and diagnosing faults is vital in UAV flight monitoring as it helps ensure the aircraft's safety, stability, and dependability.A Convolutional Neural Network (CNN) extracts features and removes noise from UAV data to diagnose actuator faults [20].Ghazali et al. [21] proposed a fault detection based on the vibration of the multirotor arms using artificial intelligence (AI).Yang et al. [22] have presented a method to detect propeller damage only based on the audio noise caused by the UAV's flight.Similarly, Liu et al. [23] also proposed to detect propeller damage using audio noise collected from the UAV's flight.CNN is then utilized to classify spectrograms as input data and allow the distinguishment of broken and unbroken propellers by applying transfer learning to various UAV testing scenarios.A study by Shibl et al. [24] proposed a proper battery management system (BMS) to increase the lifetime and efficiency of the battery.The system utilized Deep Neural Networks (DNN) and Long Short-Term Memory (LSTM) through a classification problem for the reliability of UAVs.
Machine learning (ML) tools are essential to extract hidden information from various sensor data for failure detection and identification.This project aims to develop a sound-based monitoring system for VTOL UAV flight conditions and fault identification using Machine Learning.Due to the advancement of machine learning, integrating the classification of battery performance and faulty propeller conditions is an essential factor in improving safety and efficiency across a spectrum of industries.
An extensive option of ML algorithm can be chosen for fault detection and identification depending on sensors used in the experiment and signals collected from UAV flight.Casabianca et al. [25] compare the performance of different types of deep neural networks, Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), and Convolutional Recurrent Neural Networks (CRNN), in detecting UAV faults using acoustic signals.Iannace et al. [26] built a model based on artificial neural network algorithms by measuring the noise emitted by the VTOL UAV to identify balanced and unbalanced blades in its propeller.
Vibration sensors are another instrument for gathering essential signals comparable to UAV sound data.Zhang et al. [27] proposed a UAV fault detection and identification (FDI) method based on airframe vibration signals using airborne acceleration.This study uses data from a triaxial accelerometer to detect and identify quadcopter blade faults through a Long-and Short-Term Memory (LSTM) network model.Using a similar LSTM model, Jiangmeng et al. [20] introduce a hybrid CNN-LSTM model in their study for the fault diagnosis of actuator faults.Support Vector Machine (SVM) is another widely used data classification method.Yol et al. [28] used sound based SVM for fault classification of the VTOL UAV.A study by Bondyra et al. [29] states that using SVM to determine the occurrence and character of the rotor fault can further improve the accuracy of the detection process.The signal pre-processing step is crucial for accurate and efficient sound data analysis to provide valuable information about the UAV's potential faults.Shiri et al. [30] use Variational Mode Decomposition (VMD) to remove noise from acoustic signals to detect damage in rotating machines.Rangel-Magdaleno et al. [31] use Discrete Wavelet Transform (DWT) in their study to decompose sound signals in detecting the unbalanced blade of a UAV.DWT can be used to extract useful information from a signal, as well as for denoising, compression, and feature extraction [32].Yaman et al. [33] use the Mel-frequency Cepstral Coefficients (MFCC) method for feature extraction of the audio signal in UAV motor's fault detection.Dumitrescu et al. [34] claim that the success of MFCC is due to a filter bank that uses wavelet transforms to process the Fourier Transform, which is like how the human auditory system works.

II. MATERIAL AND METHOD
Figure 2 shows the flowchart for carrying out the project.First, a study of recent research is conducted to compare and review all the methods developed by other researchers in this field.Next, we plan for experimental setup, including the test room, the VTOL UAV settings, and the microphone setup for recording audio data.After that, it will go through a recording process to collect all the sound signals.Then, all the data were pre-processed and analyzed to extract informative features.Finally, we run several classification models to classify the faults according to the respective groups.

A. Experimental Setup
This study uses DJI Mavic Pro as the main recording subject of VTOL UAVs with pre-assigned faulty mechanisms, which will be prepared with different propeller conditions.Ulanzi J12 Wireless Microphone is attached to the subject.At the same time, the microphone receiver was linked to the iPhone application device to gather the sound signals during the experiment, as shown in Figure 3. Three faulty propeller conditions were created and named Faulty 1, Faulty 2, and Faulty 3, as shown in Table I.Faulty 1 is designed for faulty propeller blades located at right counterclockwise (CCW) and left clockwise (CW) positions.Faulty 2 is created for faulty propeller blades at the right and left CCW positions.

B. Recording phase
To maximize the effectiveness of the sound signal data collection, the VTOL UAV was put up at the same height as the receiver during flight, which was 2.5 meters.The sound recording took a total of 12 minutes of each propeller condition.The 12-minute record is divided into four groups to differentiate the battery level before the battery runs out.The VTOL UAV battery has also been measured before and after every flight.Table II presents VTOL UAV battery percentage data, while Table III illustrates the sound signal collected for the three conditions to analyze the performance due to battery degradation.The raw data consists of 48,000 samples x 180 sec for each faulty condition.After pre-processing, the cleaned data were shortened into 100 segments of one-second signals.Due to significant differences in battery performance, only data from the highest battery (HB) and lowest battery (LB) groups will be used in this study to compare classification accuracy.Table IV shows the one-second sound signal of each condition.MFCC spectrum computation is implemented for the shortened sound samples to extract the informative features.

C. MFCC Feature Extraction
Mel frequency Cepstral coefficients (MFCC) technique is implemented in MATLAB (R2020a) to extract valuable features from the recorded sound signals.MFCC are very common and one of the best methods for feature extraction when talking about the 1D signals [34].The Mel frequency transform is a commonly employed technique for feature extraction from audio signals.
The block diagram depicted in Figure 4 outlines the Mel Frequency transformation process, including signals framing, windowing, FFT spectrum transformation, Mel filterbank, log transforms, discrete cosine transformation, and cepstral coefficient computation.MFCC features will be extracted for each of the three groups of VTOL UAV datasets before further classification in MATLAB Classification Learner.The plotted MFCC spectrum in Table V visually represents the extracted features.These coefficients capture the sound signal's temporal variations and represent the audio's spectral envelope.The cepstral coefficients were defined as a line or curve on the plot, showing the magnitude or intensity of each coefficient.Fig. 6 illustrates a graphical representation for comparative accuracy between three selected classifier models.As shown in Table VI, by running with 5-fold cross-validation, the best training accuracy was calculated with the LSVM, with an accuracy of 99.44% for HB and 98.89% for LB.In contrast, the MT model gained the lowest accuracy of 87.78% and 86.67% for HB and LB flights, respectively.LSVM and MT training accuracy slightly differ in classification learning ability between classifier models developed using data from HB and LB flights.The classification training algorithm developed using HB flight data produces better accuracy than LB flight data.
The accuracy was measured with other samples to test the performance of classification models.The classification model testing accuracy presented in Table VII shows that LSVM and LD are similar in classifying the faulty groups, with an accuracy of 98.33% for HB.However, the testing accuracy for the classifier model using the LB dataset shows reduced accuracy compared to the HB model.Comparison between the models using LB data shows that LD accuracy is higher than LSVM with a 0.83% difference.Lastly, the MT model is the lowest for both HB and LB model testing accuracy, with 85.83% and 82.5%, respectively.
Although the LD training algorithm using LB flight data shows slightly higher accuracy than the classification model developed using HB flight with a 0.55% difference, the training time is longer in LB than in HB flight data, as shown in Table VIII.Besides, the testing accuracy for the LD model is lower using LB flight data compared to HB with a 0.83% difference.
The obtained training and testing accuracy shows a clear difference in classification learning performance developed using data recorded from HB and LB flights.These results show that battery degradation can affect the performance of the VTOL UAV faulty classification algorithm.In terms of the methods used, Altinators et al. [4] proposed a sound-based fault identification by using a microphone fixed at a distance of about 1 meter from the UAV with an accuracy of 96.16%.In contrast, our method uses a microphone attached to a VTOL UAV, resulting in higher accuracy when using LSVM and LD models.Microphone position to collect the VTOL UAV sounds might significantly reduce noises from the surroundings.The dataset for this study was obtained within a controlled laboratory setting.Acoustic reflections can still occur in the laboratory setting, echoing sound.Sound recordings become highly challenging in outdoor settings.

IV. CONCLUSION
In conclusion, battery degradation can affect the performance of VTOL UAV's faulty classification models using sound flight data.LSVM and MT classification models developed using high-battery flight data produce better accuracy than low-battery flight data in both training and testing phases.The LD classification model developed using high-battery flight data produces better accuracy than lowbattery flight data only in the testing phase.
MFCC has proven its ability to capture sound characteristics generated by different propeller faulty conditions, which is essential for effectively classifying UAV flight conditions.This study shows promising results in classifying propeller faulty conditions for real-time flight monitoring by integrating sound sensors on UAVs.In the future, efforts can be made to enhance the reliability and accuracy of the collected data using high-performance wireless mic to reduce surrounding noises.Additionally, complementary analysis methods such as deep learning approaches can be considered to enhance faulty classification for complex and multiple faults.

Faulty 3
is created for faulty propeller blades at the left sides of both CCW and CW positions.

Fig. 3
Fig. 3 Setup of drone and microphone

Faulty 2 Faulty 3 III
Fig. 5 and Fig. 6 illustrate scatter plot graphs of the features obtained from MFCC feature extraction, which involves the 180-sample data of high and low battery levels.

Fig. 5
Fig. 5 Scatter plot graph of Linear SVM model for faulty class prediction using high-battery flight data.

Fig. 6
Fig. 6 Scatter plot graph of Linear SVM model for faulty class prediction using low-battery flight data.In this study, features extracted by MFCC for both flight times have been tested for three classifiers deploying the MATLAB classification learner tool.The three selected classifiers are Medium Tree (MT), Linear Support Vector Machine (LSVM), and Linear Discriminant (LD).The blue,

Fig. 7
Fig. 7 Percentage accuracy of three classifier model validation and test.

TABLE III RAW
DATA FROM HIGHEST (HB) AND LOWEST BATTERY (LB) LEVEL FOR

TABLE IV ONE
-SECOND SHORTENED SOUND SIGNALS FROM HIGH AND LOW BATTERY

TABLE V PLOTTED
MFCC SPECTRUM

TABLE VI CLASSIFIER
MODEL TRAINING ACCURACY