Research on the condition monitoring method of unmanned aerial vehicle based on improved multivariate state estimation technique

Abstract

The widespread use of unmanned aerial vehicles (UAVs) stimulates the demand for condition monitoring methods. To this end, a wide range of condition monitoring methods have been developed to monitor UAVs’ performance. However, since the relatively low accuracy of monitoring and high reliance on human experience for threshold setting, the performance of condition monitoring methods for UAVs is deficient. Therefore, this paper proposes an advanced condition monitoring method for UAVs which is composed of improved multivariate state estimation technique (IMSET) and a novel threshold setting method based on probability distribution. Firstly, the IMSET constructs memory matrix (MM) by dynamic selection with incremental learning to improve the accuracy of estimation. Secondly, the exponentially weighted moving average (EWMA) is employed to mitigate the impact of measurement errors in condition vectors and then the threshold is set by probability distribution to reduce the dependence on human experience. To verify the effectiveness of the proposed method, sufficient experiments based on condition monitoring data generated by DJI F450 are conducted. The experimental results demonstrate that the method proposed in this paper can accurately monitor the condition of UAVs in time.

Introduction

With the widespread use of UAVs, the accident frequency of UAVs caused by wear and tear or improper human operation is gradually increasing. The main faults that cause accidents of UAVs include motor and battery failures, arm or propeller breakage, and environmental factors¹. To effectively decrease the accident frequency, it is necessary to monitor the condition of UAVs in real-time.

At present, condition monitoring methods are mainly divided into three categories, which are model based methods², expert system-based methods^3,4,5, and data-driven based methods. The model-based methods are with relatively low generalization among different devices. The method based on expert system highly relies on human experience, which is strongly subjective and has a large data collection workload. With the development of machine learning technology, data-driven methods have been widely applied in the field of condition monitoring and fault detection^6,7. Based on data-driven methods, by learning the features of existing historical data, it is possible to monitor the condition of UAVs. Li et al.⁸ used the Broad Learning System (BLS) to achieve accurate condition monitoring and rapid faults diagnosis. Fernández-Bobadilla et al.⁹ provided a review of railway condition monitoring algorithms of data-based, model-based, and hybrid methods. M. Bronz et al.¹⁰ monitored and diagnosed faults in the actual flight process of UAV based on support vector machines (SVM) combined with feature trajectories.

The above methods require a large number of normal and abnormal data to build the training set. However, the low frequency and short duration of UAVs’ malfunctions result in the collection of abnormal data is far less than normal data. The disparity in data quantity between normal and abnormal data makes it difficult to establish a training set containing enough real abnormal data. In response to this issue, a normal behavior model (NBM) can be constructed to estimate the current condition¹¹. At present, there are some condition monitoring methods based on NBM. Zhang et al.¹² proposed a condition monitoring method based on NBM by gaussian mixture models (GMM). Wang et al.¹³ monitored the condition of UAVs by multivariate regression and long short-term memory (LSTM), and achieves fault detection and recovery. Gross et al.¹⁴ developed multivariate state estimation technique (MSET) to assure the continued safety of nuclear power station. Guo et al.¹⁵ constructed the NBM for wind turbines through MSET and artificial neural networks (ANNs), and achieved condition monitoring. Wang et al.¹⁶ proposed a memory matrix (MM) construction method by k-nearest neighbors (KNN) and achieved condition monitoring and fault detection. Zhang et al.¹⁷ proposed an improved MSET that employs probability density uniformly-spaced sampling to construct MM and achieves condition monitoring by similarity-threshold. Wang et al.¹⁸ studied condition monitoring of wind turbine gearbox by MSET. Lv et al.¹⁹ proposed an improved MSET by discrete particle swarm optimization (DPSO) to achieve early fault warning for induced draft. Yan²⁰ combined sequential probability ratio with MSET for condition monitoring and fault diagnosis of UAVs.

Compared to the above-mentioned methods, the MSET constructs NBM by memory matrix (MM) and obtains estimated data through nonlinear calculation. There is no requirement of complex calculations and model training processes for MSET, which can reduce computational complexity and effectively avoids time loss caused by model training. Therefore, the MSET is widely employed in NBM construction^21,22,23,24.

However, existing condition monitoring methods based on NBM suffer from relatively low accuracy of monitoring and high reliance on human experience for threshold setting. In response to this issue, this paper proposes a novel condition monitoring method based on improved multivariate state estimation technique (IMSET) and a threshold setting method based on probability distribution. This method includes four key steps, including MSET model construction, dynamic memory matrix (DMM) construction, similarity with EWMA, and threshold setting. Firstly, the conventional MSET model is constructed. Secondly, the MSET is improved by DMM, which is constructed by dynamic selection and incremental learning. The incremental learning is employed to learn more data during the normal operation of UAVs, improving and perfecting the MM. Thirdly, the similarity sequence with EWMA is calculated to mitigate the impact of measurement errors in condition vectors. Fourthly, the threshold is set based on probability distribution of similarity sequence.

The main contributions of this paper are as follows:

• (1) The MSET model is improved by MM dynamic selection and incremental learning. The IMSET model can estimate more accurately and is more suitable for UAV’s condition monitoring.

• (2) The EWMA is employed for smoothing similarity sequence, which effectively decreases the misdiagnosis frequency and monitors abnormal condition of UAVs in time.

• (3) A threshold setting method based on probability distribution is proposed, which reduces the dependence on human experience.

The rest of this paper is organized as follows. Section 2 provides details regarding the proposed method. Section 3 shows the experimental results to verify effectiveness of the proposed condition monitoring method for UAVs. The section 4 concludes this paper.

Methodology

The proposed methodology consists of five main sections: 1) multivariate state estimation technique, 2) dynamic memory matrix construction, 3) exponentially weighted moving average similarity, 4) threshold setting, and 5) proposed condition monitoring method.

Multivariate state estimation technique

Multivariate state estimation technique (MSET) is a non-parametric, non-linear, and feature-driven method for analyzing multivariate time series. MSET collects sufficient historical data that encompasses the normal conditions of the system and learns to identify empirical patterns through this process. Subsequently, The MSET calculates the corresponding estimated vector for each observed vector based on this empirical pattern. The MSET flowchart is illustrated in Figure 1.

Fig. 1

MSET flowchart.

Next, the MSET modeling process will be introduced in detail. Firstly, construct the MM D which is composed of the condition vector x, representing monitoring data of UAVs at a specific time point. The MM D with n features and m samples is shown in equation (1).

$$\begin{aligned} \begin{array}{l} {{\varvec{D}}} = \left[ {{{\varvec{x}}}(1),{{\varvec{x}}}(2), \cdots ,{{\varvec{x}}}(m)} \right] \\ = {\left[ {\begin{array}{*{20}{l}} {{x_1}(1)}& {{x_1}(2)}& \cdots & {{x_1}(m)}\\ {{x_2}(1)}& {{x_2}(2)}& \cdots & {{x_2}(m) }\\ { \vdots }& { \vdots }& \ddots & { \vdots }\\ {{x_n}(1)}& {{x_n}(2)}& \cdots & {{x_n}(m)} \end{array} } \right] _{n \times m}} \end{array} \end{aligned}$$

(1)

Among them, n represents the condition vector x contains n features, and m represents D has m condition vectors.

The variable ${{{\varvec{x}}}_{obs}}$ represents the observed condition vector of the device and the variable ${{{\varvec{x}}}_{est}}$ represents the corresponding estimated condition vector. ${{{\varvec{x}}}_{est}}$ can be calculated by the following equation (2).

$$\begin{aligned} \begin{array}{l} {{{\varvec{x}}}_{est}} = {{\varvec{D}}} \cdot {{\varvec{w}}} = \left[ {{{\varvec{x}}}(1),{{\varvec{x}}}(2), \cdots ,{{\varvec{x}}}(m)} \right] \left[ \begin{array}{l} {w_1}\\ {w_1}\\ \vdots \\ {w_m} \end{array} \right] \\ = {w_1} \cdot {{\varvec{x}}}(1) + {w_2} \cdot {{\varvec{x}}}(2) + \cdots + {w_m} \cdot {{\varvec{x}}}(m) \end{array} \end{aligned}$$

(2)

In equation (2), w is the weight vector required for ${{{\varvec{x}}}_{est}}$ calculation. Set residual vector $\varvec{\varepsilon } = {{{\varvec{x}}}_{obs}} - {{{\varvec{x}}}_{est}}$, measuring the degree of difference between the estimated condition vector and observed condition vector. S(w) represents the sum of squares of the residual vector $\varvec{\varepsilon }$, and its formula is shown in equation (3).

$$\begin{aligned} \begin{array}{l} S\left( {{\varvec{w}}} \right) = \sum \limits _{i = 1}^n {{\varvec{\varepsilon } _i}^2} = {\varvec{\varepsilon } ^T}\varvec{\varepsilon } = {\left( {{{{\varvec{x}}}_{obs}} - {{{\varvec{x}}}_{est}}} \right) ^T} \cdot \left( {{{{\varvec{x}}}_{obs}} - {{{\varvec{x}}}_{est}}} \right) \\ = {\left( {{{{\varvec{x}}}_{obs}} - {{\varvec{D}}}{{\varvec{w}}}} \right) ^T} \cdot \left( {{{{\varvec{x}}}_{obs}} - {{\varvec{D}}}{{\varvec{w}}}} \right) = {\sum \limits _{i = 1}^n {\left( {{{{\varvec{x}}}_{ob{s_i}}} - \sum \limits _{j = 1}^m {{{{\varvec{w}}}_j}{{{\varvec{D}}}_{ij}}} } \right) } ^2} \end{array}. \end{aligned}$$

(3)

Taking the partial derivative of ${{\varvec{w}}}_{k}, k \in [1,m]$ and set it as 0, the equation (4) is derived.

$$\begin{aligned} \frac{{\partial S({{\varvec{w}}})}}{{\partial {{{\varvec{w}}}_k}}} = - 2\sum \limits _{i = 1}^n {\left( {{{{\varvec{x}}}_{ob{s_i}}} - \sum \limits _{j = 1}^m {{{{\varvec{w}}}_j}{{{\varvec{D}}}_{ij}}} } \right) } \cdot {{{\varvec{D}}}_{ik}} = 0 \qquad k \in [ {1,m}] \end{aligned}$$

(4)

After simplification, the equation (4) can be transformed into the matrix form of the following equation (5).

$$\begin{aligned} {{{\varvec{D}}}^T} \cdot {{{\varvec{x}}}_{obs}} = {{{\varvec{D}}}^T}{{\varvec{D}}} \cdot {{\varvec{w}}} \end{aligned}$$

(5)

Through multiplying by matrix ${\left( {{{{\varvec{D}}}^T}{{\varvec{D}}}} \right) ^{ - 1}}$ in the both sides of equation (5), the weight vector w is obtained. However, when the ${\left( {{{{\varvec{D}}}^T}{{\varvec{D}}}} \right) }$ is singular matrix, the calculation of the weight vector w mentioned above will result in errors. To avoid algorithm errors caused by it, a nonlinear operator $\oplus$ is employed to derive the final expression of the weight vector w, as shown in the following equation (6).

$$\begin{aligned} {{\varvec{w}}} = {\left( {{{{\varvec{D}}}^T} \oplus {{\varvec{D}}}} \right) ^{ - 1}} \cdot \left( {{{{\varvec{D}}}^T} \oplus {{{\varvec{x}}}_{obs}}} \right) \end{aligned}$$

(6)

By substituting equation (6) into equation (2), the formula for estimated condition vector ${{\varvec{x}}}_{est}$ can be derived, as shown in equation (7).

$$\begin{aligned} {{{\varvec{x}}}_{est}} = {{\varvec{D}}} \cdot {{\varvec{w}}} = {{\varvec{D}}} \cdot {\left( {{{{\varvec{D}}}^T} \oplus {{\varvec{D}}}} \right) ^{ - 1}} \cdot \left( {{{{\varvec{D}}}^T} \oplus {{{\varvec{x}}}_{obs}}} \right) \end{aligned}$$

(7)

In this paper, the nonlinear operator $\oplus$ represents the Euclidean distance calculation. Consequently, the formulas for deriving ${{{\varvec{D}}}^T} \oplus {{\varvec{x}}}_{obs}$ and ${{{\varvec{D}}}^T} \oplus {{\varvec{D}}}$ are as shown in the following equation (8) and equation (9).

$$\begin{aligned} {{{\varvec{D}}}^T} \oplus {{{\varvec{x}}}_{obs}} = {\left[ \begin{array}{l} \sqrt{\sum \limits _{i = 1}^n {{{[{x_i}(1) - {{{\varvec{x}}}_{obs}}_{i}]}^2}} } \\ \sqrt{\sum \limits _{i = 1}^n {{{[{x_i}(2) - {{{\varvec{x}}}_{ob{s_i}}}]}^2}} } \\ \vdots \\ \sqrt{\sum \limits _{i = 1}^n {{{[{x_i}(m) - {{{\varvec{x}}}_{ob{s_i}}}]}^2}} } \end{array} \right] _{m \times 1}} \end{aligned}$$

(8)

$$\begin{aligned} {{{\varvec{D}}}^T} \oplus {{\varvec{D}}} = {\left[ {\begin{array}{*{20}{c}} 0& {\sqrt{\sum \limits _{i = 1}^n {{{[{x_i}(1) - {x_i}(2)]}^2}} } }& \cdots & {\sqrt{\sum \limits _{i = 1}^n {{{[{x_i}(1) - {x_i}(m)]}^2}} } }\\ {\sqrt{\sum \limits _{i = 1}^n {{{[{x_i}(2) - {x_i}(1)]}^2}} } }& 0& \cdots & {\sqrt{\sum \limits _{i = 1}^n {{{[{x_i}(2) - {x_i}(m)]}^2}}} }\\ \vdots & \vdots & \ddots & \vdots \\ {\sqrt{\sum \limits _{i = 1}^n {{{[{x_i}(m) - {x_i}(1)]}^2}} } }& {\sqrt{\sum \limits _{i = 1}^n {{{[{x_i}(m) - {x_i}(2)]}^2}} } }& \cdots & 0 \end{array}} \right] _{m \times m}} \end{aligned}$$

(9)

Dynamic memory matrix construction

The scale and construction method of memory matrix (MM) are critical factors that influence the accuracy and computational efficiency of the MSET model. The scale refers to the number of condition vectors in MM, denoted by sample number m in equation (1), which is often determined experimentally. Generally, larger scale means better accuracy but lower efficiency. After scale setting, the MM needs to be constructed.

In existing researches, there are two primary construction methods for MM, including static memory matrix (SMM) and dynamic memory matrix (DMM). SMM no longer changes after construction. DMM is a concept opposite to SMM, which can be adjusted after construction. DMM effectively addresses the limitations of SMM. In existing research, there are mainly two types of construction methods for DMM, namely the updating method and the selection method. The updating method begins with sampling original MM, and then updates the original MM. The selection method constructs DMM by selecting condition vectors from the training set based on observed condition vector. The differences between SMM and DMM construction are illustrated in Figure 2.

Fig. 2

The differences between SMM and DMM construction.

In existing research, scholars have studied the different MM construction for MSET^{14,15,16,17,18,19,20}, and the description of each method is shown in Table 1.

Table 1 MM description of existing MEST models

Although the existing MM construction method has made some improvements, there are still shortcomings. The accuracy of MSET^14,17,18,19 based on SMM tends to decrease when the flight missions of UAVs changed, because the SMM will not change after original construction. Moreover, the accuracy of MSET^15,20 based on DMM updating is relatively low because the uniformly-spaced sampling of original MM is with randomness and the update level is limited. The uniformly-spaced sampling involves taking samples at a fixed interval l. Variations in the interval or the starting point may lead to changes in MM construction. The process of the uniformly-spaced sampling is illustrated in Figure 3. Furthermore, due to the variability in monitoring data generated from different conditions and environment, the incremental learning of UAV’s monitoring data during normal conditions is necessary. However, the MSET¹⁶ based on DMM selection neglects incremental learning, the MM constructed may not adequately adapt to changes in the UAV’s flight missions, which will lead to accuracy decrease of MSET.

Fig. 3

The process of uniformly-spaced sampling.

In response to the aforementioned limitations, this paper proposes an improved DMM selection method based on clustering and incremental learning of training set data. By clustering the training set, this method reduces the redundance of training set and increases computational efficiency during DMM construction. Furthermore, by incorporating the training set incremental learning, the MM is able to adequately adapt to changes in UAV’s flight missions, leading to an enhancement in accuracy of the MSET model. A detailed introduction to the process of DMM selection in this paper is as follows:

(1) Reduce redundance of the training set by k-means clustering method and select one condition vector from each cluster to get the new training set. Subsequently, the original training set is replaced by this new set. The process is illustrated in Figure 4.

Fig. 4

Training set clustering process.

(2) Select the MM D from the training set, and the process is as follows.

1) Calculate the Euclidean distance d between the observed condition vector ${{\varvec{x}}}_{obs}$ and each condition vector x in the training set. The calculation formula of Euclidean distance d is shown in equation (10).

$$\begin{aligned} d = \sqrt{\sum \limits _{i = 1}^n {{{({{{\varvec{x}}}_{i}} - {{\varvec{x}}_{obs}}_{i})}^2}} } \end{aligned}$$

(10)

2) Select m pieces of x with the smallest Euclidean distance d from the training set to construct D. The pseudo-code of the proposed DMM sampling algorithm is shown below.

Algorithm 1

DMM-sampling.

(3) Substitute the obtained MM into the MSET model to monitor conditions of UAVs. The incremental learning process of the training set is illustrated in Figure 5. If the observed vector, derived from the UAV’s flight logs, is deemed normal, the action ”Add observed vector” is executed, and the observed vector is added to the clustered training set. Conversely, if the condition is abnormal, the vector is not added.

Fig. 5

Incremental learning process of the training set.

Through the processes mentioned above, the IMSET model is constructed. The DMM construction method proposed in this section is expected to enhance the accuracy of the IMSET model and increase computational efficiency. Consequently, the estimated condition vector calculated by the IMSET model will be more accurate, which is beneficial to condition monitoring for UAVs.

Exponentially weighted moving average similarity

Relying solely on the IMSET model is insufficient for achieving real-time condition monitoring of UAVs. Therefore, after obtaining the estimated condition vectors by IMSET, it is necessary to calculate the difference between the estimated vector and observed vector. This section employs a similarity with exponentially weighted moving average (EWMA) to describe the difference, and then proposes a threshold setting method based on the probability distribution of similarity sequence to reduce the dependence on human experience. By comparing the similarity with threshold, the condition of UAVs can be monitored.

The indicator namely similarity is used to quantify the difference between multidimensional observed and estimated condition vectors of UAVs. For the observed condition vector ${{\varvec{x}}}_{obs}$ and estimated condition vector ${{\varvec{x}}}_{est}$ of UAVs, the formula for similarity s is shown in equation (11).

$$\begin{aligned} s = \frac{1}{{1 + dist({{{\varvec{x}}}_{est}} - {{{\varvec{x}}}_{obs}})}} \end{aligned}$$

(11)

In equation (11), $dist({{{\varvec{x}}}_{est}} - {{{\varvec{x}}}_{obs}})$ represents the Euclidean distance between estimated condition vector and observed condition vector.

In practical situations, the sensors of UAVs may experience malfunctions due to the uncertain factors and random interference^25,26. These malfunctions may result in measurement errors in the observed condition vectors ${{\varvec{x}}}_{obs}$, leading to errors of similarity s calculation, and further resulting misdiagnosis in condition monitoring. Therefore, the EWMA is employed to smooth the similarity sequence, thereby reduce misdiagnosis²⁷. The EWMA is expressed as follows.

$$\begin{aligned} MA{S_t} = \frac{{{s_t} + \alpha {s_{t - 1}} + {\alpha ^2}{s_{t - 2}} + \ldots + {\alpha ^{L - 1}}{s_{t - L + 1}}}}{{1 + \alpha + {\alpha ^2} + \ldots + {\alpha ^{L - 1}}}} \end{aligned}$$

(12)

In equation (12), $s_t$ and $MA{S_t}$ represent similarity and similarity with EWMA at time point t respectively, the variable L represents the length of moving window, and $\mathrm{{0 < }}\alpha \le 1$ is the smoothing factor. Compared with traditional moving average (MA), the EWMA gives more weight to recent observed vectors, resulting in that the similarity sequence with EWMA is better to reflect the actual condition of UAVs. Therefore, the similarity sequence with EWMA not only reduces misdiagnosis, but also achieves real-time condition monitoring.

Threshold setting

The threshold is used to compare with the similarity sequence. If the similarity falls below the threshold, the condition of the UAV is deemed abnormal, and an abnormal condition warning is triggered.

In existing research, the threshold is commonly obtained by directly setting or by calculating the mean and standard deviation^16,17,20,27. These methods require manual setting of key parameters and rely heavily on human experience. However, human experience is sometimes highly unstable and subjective. To reduce reliance on human experience, this section proposes a threshold setting method based on the probability distribution of the similarity sequence. Generally, the probability density function (PDF) and cumulative distribution function (CDF) are used to describe probability distribution. The relationship between PDF and CDF is illustrated in Figure 6.

Fig. 6

The relationship between PDF and CDF.

Subsequently, the process of the threshold setting is shown as follows.

• (1) Establish a validation set composed of normal condition vectors of UAVs, and calculate the similarity sequence of the validation set by the IMSET model.

• (2) Calculate the PDF and CDF of similarity sequence. Set the threshold as $\textit{E}_\textit{T}$, while the CDF value of the interval $\mathrm{{[}}{E_T}\mathrm{{,1]}}$ is sufficiently large, the interval can be considered as the normal value interval of the similarity.

• (3) Calculate the threshold $\textit{E}_\textit{T}$ by the following equation (13).

  $$\begin{aligned} p = P({E_T} \le x \le 1) = \int _{{E_T}}^1 {f(x)dx} \end{aligned}$$

  (13)

In equation (13), p represents the probability that the similarity sequence falls within normal value interval. In this paper, p is set as 0.995. Moreover, f(x) is the PDF of the similarity sequence, and x represents the similarity value. It is necessary to note that, when there is no access to obtain the function f(x), the threshold $\textit{E}_\textit{T}$ can be obtained by the CDF figure. The specific process will be explained in the next section. Compared to the common threshold setting methods, the proposed method reduces the dependence on expert knowledge and human experience, offering broader applicability.

Proposed condition monitoring method

This section has described the four key steps of proposed condition monitoring method, including MSET model construction, DMM construction, exponentially weighted moving average similarity, and threshold setting. By putting the four steps together, the condition monitoring method is proposed. Furthermore, the Figure 7 shows the relationship of the above-mentioned four key steps and the processes of proposed condition monitoring method.

Fig. 7

The framework of proposed condition monitoring method.

The offline section collects data from the normal condition of the UAV and divides it into a training set and a validation. The training set is generated from raw training set through clustering and is then used for DMM construction and IMSET modeling. The validation set is used to calculate similarity sequence and set the threshold. The online section collects condition data from the UAV as a test set. Subsequently, the online section calculates the similarity sequence with the EWMA and compares it with the threshold, thereby to achieve condition monitoring of UAVs. It should be noted that clustering is not required for the test set, as each data point is directly input into the IMSET model to calculate the similarity between the actual and predicted data. The increment section adds the observed condition vector of the UAV to the training set when the condition is normal.

Experiment and results

The section of experiment and results is composed of four main sections: 1) data acquisition and processing, 2) memory matrix scale setting, 3) verification of IMSET model, and 4) test of condition monitoring method. Specifically, the first two sections provide data, parameters, and metrics for subsequent experiments. The third section validates the accuracy of the IMSET model, and the last section tests the effectiveness of proposed condition monitoring method.

Data acquisition and processing

The research object of this case is the DJI F450. The total mass of the UAV is approximately 1.5 kg, and the radius of its body is about 0.225 m. In this paper, the data is collected from the flight logs, which are transmitted via a communication link from the UAV to the ground station. Specifically, the DJI F450 in this paper executes waypoint flight missions. During the waypoint flight mission, the UAV follows a series of pre-set waypoints and performs various flight modes such as hovering, cruising, and turning. This paper analyzes the flight logs generated during the flight mission of DJI F450 and collects its condition monitoring data, which represent the conditions of UAVs. Based on the summary of references^8,10, the UAV’s condition monitoring parameters selected in this paper are shown in Table 2.

Table 2 condition monitoring parameters for UAVs

The three attitude angles and angular accelerations represent the UAV’s flight attitude, while the actuator control signals represent the UAV’s actuator condition. Therefore, the selected parameters reflect the operation state of both the UAV and its actuators. The condition vector is composed of selected monitoring parameters, which is shown in equation (14).

$$\begin{aligned} \begin{array}{l} x = [angleR,angleP,angleY,R\_acc,P\_acc,Y\_acc, PWM1,PWM2,PWM3,PWM4{]^\mathrm{{T}}} \end{array} \end{aligned}$$

(14)

After determining the condition monitoring parameters, the next step is to construct the dataset utilized in this experiment. A total of 13 flight logs were meticulously collected from DJI F450, yielding 13 sets of condition monitoring data. Among these datasets, 11 groups are generated during normal flight. The first 10 groups are designated as the training set, with the 11th group serving as the validation set. The 12th and 13th groups are generated by DJI F450 during malfunctioning flight missions, which are designated as the test set 1 and test set 2 respectively. The experimental datasets are illustrated in Figure 8.

In detail, the training set, validation set, test set 1, and test set 2 encompass 16532, 3194, 1473, and 891 pieces of monitoring data respectively. Furthermore, the training set and validation set only comprise normal monitoring data. The test set 1 comprise 1322 normal and 151 abnormal monitoring data ,while the test set 2 is composed of 719 normal and 172 abnormal monitoring data. It is noteworthy that the time interval between each piece of data in the flight logs is approximately 0.5 seconds. Consequently, the training set data covers over 2 hours of flight logs generated by the DJI F450 across 10 flights, which is sufficient enough for the IMSET modeling.

Fig. 8

Separation of experimental datasets.

Memory matrix scale setting

Due to the importance of the memory matrix scale, this section undertakes a comprehensive assessment, weighing both accuracy and computational efficiency, to determine the optimal MM scale. This paper employs two metrics, namely the root mean square error (RMSE) and the mean absolute percentage error (MAPE), to evaluate the accuracy of condition monitoring. Furthermore, the computational time is used to evaluate the efficiency of condition monitoring. Specifically, RMSE and MAPE are calculated by equation (15) and equation (16), respectively.

$$\begin{aligned} RMSE = \sqrt{\frac{1}{n}\sum \limits _{i = 1}^n {{{\left( {{y_i} - {{\tilde{y}}_i}} \right) }^2}} } \end{aligned}$$

(15)

$$\begin{aligned} MAPE = \frac{1}{n}\sum \limits _{i = 1}^n {\left\| {\frac{{{y_i} - {{\tilde{y}}_i}}}{{{y_i}}}} \right\| } \end{aligned}$$

(16)

To enhance the credibility of the experiment, this section evaluates the RMSE, MAPE, and computational time of the MSET model^14,15,16 and the IMSET model with different memory matrix scale. The experiment is based on the normal data of test set 1, and the results are shown in Table 3-5 and Fig 9.

Table 3 RMSE of different MM scale.

Table 4 MAPE of different MM scale.

Table 5 Computational time of different MM scale.

Fig. 9

The impact of MM Scale on MSET Models.

From Table 3-5 and Fig 9, it is observed that the scale of the memory matrix significantly impacts the RMSE, MAPE, and computational time of the MSET model. Specifically, when the scale exceeds 100, the computational time increases substantially, while the RMSE and MAPE decrease slowly. Conversely, when the scale is below 100, the computational time increases slowly, but the RMSE and MAPE decrease rapidly. Considering both the accuracy and efficiency of the MSET model, this paper sets the optimal scale of the memory matrix as 100.

Verification of IMSET model

This section assesses the performance of the IMSET model proposed in this paper. The MSET models mentioned in Table 1 are used for comparison. Furthermore, to enhance the reliability of results, more recent condition monitoring methods are employed for comparison, such as LSTM¹³ and BiGRU²⁸. The network structure of LSTM and BiGRU are shown in Table 6. This experiment initially employs normal data of the test sets to verify the accuracy of the model, and subsequently employs overall test sets to verify if the IMSET model can accurately identify normal and abnormal condition of UAVs.

Table 6 The network structure of LSTM and BiGRU.

The experiment employs normal data of the test set 1 and test set 2 respectively to evaluate the IMSET model. And the results are shown in Table 7 and Table 8.

Table 7 Experiment result of test set 1.

Table 8 Experiment result of test set 2.

The tables show that for test set 1, the RMSE and MAPE of the IMSET model are 0.0284 and 3.44%. Specifically, compared with the LSTM, BiGRU, MSET^{14,15,16,17,18,19,20}, the MAPE of the IMSET model is decreased by 69.15%, 72.66%, 73.56%, 63.25%, 22.70%, 78.77%, 94.74%, 71.92%, and 26.50% respectively. Meanwhile, for test set 2, the RMSE and MAPE of the IMSET model are 0.0546 and 4.98%. Specifically, compared with the LSTM, BiGRU, MSET [14]-[20], the MAPE of the IMSET model is decreased by 54.93%, 53.37%, 58.77%, 48.50%, 31.31%, 80.77%, 84.41%, 53.89%, and 5.86% respectively.

The results demonstrate the accuracy of the IMSET model is greater than other methods mentioned above. It should be noted that though the computational time of the IMSET model is relatively long, it is still acceptable for processing a single piece of data. Specifically, for the test set 1 with 1322 pieces of normal condition vectors, it takes 11.53 seconds, which means the computational time for each condition vector is approximately 0.009s. Meanwhile, the computational time for each condition vector in test set 2 is also approximately 0.009s.

To further demonstrate the effectiveness of the IMSET model, this section employs overall test set 1 and test set 2 respectively to plot the fitting diagram of monitoring parameter PWM1. The results are illustrated in Figure 10 and Figure11, respectively.

Fig. 10

The fitting diagrams of test set 1.

Fig. 11

The fitting diagrams of test set 2.

Figure 10 shows that the fitting effectiveness of LSTM, BiGRU, MSET¹⁶, and IMSET are better than others. However, when the time point is greater than 1322, which means the condition of the UAV is becoming abnormal, the LSTM, BiGRU can fit the abnormal data well, which indicates the two methods can’t distinguish between normal and abnormal condition of UAVs. Furthermore, according to Table 7, the accuracy of IMSET is greater than MSET¹⁶, so IMSET is optimal in the research. It should be noted that the error increases at time point 1332 in Figure 10(j). This is because the IMSET model is trained solely on normal data, and as a result, it cannot accurately predict abnormal data. Therefore, the prediction error increases significantly when a fault occurs at time point 1332. This demonstrates that the IMSET model is capable of detecting abnormal states when faults arise. Meanwhile, Figure 11 shows the same information. So far, this section proves that the estimation accuracy of IMSET is higher than the state-of-the-art methods mentioned in this section.

Test of condition monitoring method

This section tests the effectiveness of proposed condition monitoring method. To begin with, the threshold should be set, and the calculation steps are as follows.

(1) Calculate similarity s by equation (11) and get the similarity sequence.

(2) Calculate the threshold ${E_T}$ by equation (13). In this experiment, p is set as 0.995. So, the threshold ${E_T} = 0.990$ can be obtained. While the function f(x) is not available, the threshold $\textit{E}_\textit{T}$ can be obtained by the CDF figure, which is illustrated in Figure 12. The blue curve represents CDF, and the horizontal and vertical axis of the curve represents similarity and cumulative probability respectively. The $p = 0.995$ represents the probability of similarity being within the normal range, so the probability being within abnormal range is 0.005. When the cumulative probability is 0.005, the similarity of CDF curve is the threshold, with a value of 0.990.

Fig. 12

The CDF diagram of the similarity sequence for validation set.

Furthermore, this section tests the proposed condition monitoring method using test set 1 and test set 2. By analyzing the flight logs generated by DJI 450, the test set 1 and test set 2 have 1473 and 891 condition vectors respectively. For test set 1, the UAV is out of order at time point 1323. And for test set 2, the UAV is out of order at time point 720. Calculate the similarity with EWMA of test set 1 and test set 2 by equation (11) and equation (12), and compare them with the threshold $\textit{E}_\textit{T}$ respectively. If the similarity is less than the threshold, the condition of the UAV is abnormal. To enhance reliability of the experiment, the similarity with conventional moving average (MA) is set for comparation.

This section evaluates the condition monitoring method with test set1 and test set 2, and the results are illustrated in Fig 13 and Fig 14, respectively.

Fig. 13

The condition monitoring result of test set 1.

Fig. 14

The condition monitoring result of test set 2.

From Figure 13(a) and Figure 14(a), it is observed that the similarity sequence fluctuates violently, and there are several misdiagnoses within the normal range of time points, as highlighted in the red boxes. In contrast, the similarity with EWMA and MA are smoother, causing no misdiagnoses. It indicates that the condition monitoring method with EWMA and MA can both mitigate the impact of measurement errors caused by uncertain factors and random interference encountered during flight missions of UAVs. For further analysis, the Figure 13(b) shows that the similarity sequence with EWMA and MA start to fall below the threshold at time point 1324 and 1345 respectively. In the same way, the Figure 14(b) show that the similarity sequence with EWMA and MA start to fall below the threshold at time point 721 and 727 respectively. The further analysis indicates that the similarity with EWMA can achieve condition monitoring of UAVs in time, while the similarity with MA cannot.

In summary, this section firstly conducts preliminary experiments to provide data, parameters, and metrics. Secondly, this section fully validates the accuracy of IMSET model. Finally, this section tests the effectiveness of the proposed condition monitoring method and demonstrates that it can not only monitor the conditions of UAVs in time, but also reduce misdiagnoses.

Discussion

This paper advocates for a condition monitoring method based on IMSET and similarity with EWMA for UAVs. In particular, the proposed method as a whole comprises the following four main parts:

(1) An improved MSET model which is constructed by DMM.

(2) A similarity sequence with EWMA which is employed to compare with threshold, thereby to detect whether the operation of UAVs is normal or abnormal.

(3) A threshold setting method based on probability distribution, reducing the dependence on human experience.

(4) The proposed condition monitoring method

To the two unsolved issues mentioned in the introduction, this paper not only further improves the accuracy of condition monitoring method by IMSET model and EWMA, but also provide a threshold setting method based on probability distribution to decrease the dependence on human experience.

The experiment results of test set 1 and test set 2 prove the effectiveness of the condition monitoring method for UAVs. For test set 1 and test set 2, the MAPE of IMSET are 3.44% and 4.98% respectively, both better than the state-of-the-art methods mentioned in this paper. Furthermore, the two test sets demonstrate that the condition monitoring method with EWMA not only monitor the conditions of UAVs in time, but also reduce misdiagnoses.

In the future, the condition monitoring method will be tested on different types of UAVs. Meanwhile, the proposed condition monitoring method will be compared with more methods to further verify its effectiveness