Improved railway track faults detection using Mel-frequency cepstral coefficient and constant-Q transform features

Abstract

Regular inspection of the health of railway tracks is crucial to maintaining reliable and safe train operations. Some factors including cracks, rail discontinuity, ballast issues, burn wheels, super-elevation, loose nuts and bolts, and misalignment developed on the railways due to pre-emptive investigations, non-maintenance, and delay in detection pose grave threats and danger to the safe operation of railway transportation. In the past, manual inspection was performed for the rail track by a rail cart which is both prone to error and inefficient due to human biases and error. Several train accidents are reported in Pakistan; it is important to automate these techniques to avoid such train accidents for the safety of countless lives. This study aims to enhance railway track fault detection using an automatic rail track fault detection technique with acoustic analysis. Moreover, the proposed method contributes to making the dataset large by using the CTGAN technique. Results show that acoustic data may help to determine the railway track faults effectively and logistic regression is used to perform the classification for railway track faults with an accuracy of 100%.

Introduction

The railway network is a highly essential transportation conduit in various developing nations, such as Pakistan, and is utilized to satisfy public transit demands. The railway structure is crucial for trade and supply networks¹. The railway market has gotten stronger, opening up new opportunities for the country’s public and economy. According to², the railway industry’s annual report from 2016 to 2018 showed a growth rate ranging from 1.3% to 2.4%. As a result, high-performance railway operations are essential to ensure that the railway runs continuously and that passengers are safe. The railway system is getting more burdened and complex as the number of train passengers grows. According to³, mechanical forces and environmental factors accelerate the deterioration of train rails. The railway tracks are crucial components of the railway network. Rail track inspection helps decrease accidents, injuries, and deaths⁴. From 2013 to 2020, the registered train accidents were 127 due to rail track faults according to the annual reports in Pakistan⁴. People including students, tourists, and commuters use trains for traveling in Pakistan. From 2012 to 2017, a total of 757 train accidents have been reported⁵. Additionally, 22 goods and 16 passenger trains were derailed in 2014, and 37 goods and 37 passenger train accidents were reported in 2015. In 2019, 11434 railway accidents were recorded, causing 937 casualties and 7730 injuries⁶. However, the train accident ratio is higher in under-developing countries⁷. The railway network contributes to the Pakistani economy as it has a huge network in the North-South corridor that links the seaport of Karachi with the country’s main production centers and population⁸. In 2020, 152 train accidents are reported causing 19 deaths⁹. In 2021, 32 casualties and 64 injuries were recorded in railway accidents⁹.

Timely detection and proper inspection of faults may protect several human lives and reduce the financial losses for railway systems¹⁰. However, the maintenance and inspection of railway tracks is a time-consuming and expensive activity. Several non-destructive evaluation (NDE) methods for railway track inspection have been applied including detection using phased array technology¹¹. Eddy current testing¹², guided wave detection¹³, ultrasonic testing¹⁴, and other techniques have been the focus of rail track inspections. However, there has been a recent surge in enthusiasm for employing machine learning models, the Internet of Things (IoT), and deep learning networks in these inspections. These advanced technologies aim to enhance the speed, precision, uniqueness, and overall success of non-destructive evaluation (NDE) approaches. The integration of acoustic transducers¹⁵ and high-speed cameras¹⁶ with machine learning classifiers is becoming common to modernize traditional inspection methods. In particular, hand-crafted feature engineering (HCFE) has been utilized in audio and image-based machine learning applications. Nevertheless, HCFE demands domain-specific expertise, extensive problem-solving, and system modifications to optimize performance (PS18). Moreover, railway track classification and inspection have three main stages. Firstly, preprocessing of ‘wav’ files is performed to eliminate the undesired sounds. Secondly, feature extraction is performed with spectrograms. Thirdly, the classification method is trained to detect rail track faults.

Maintaining a reliable and safe rail network demands uninterrupted train operations, which entails extensive monitoring of hundreds of thousands of kilometers of track. This endeavor requires substantial investments of time and money. Timely and sufficient maintenance of railway tracks is crucial; any failures can disrupt train services, leading to potential financial and human consequences¹⁷. Crack identification is very important to run the system rapidly and efficiently. In Pakistan, track inspection is currently conducted using a railway cart, where human specialists manually assess the track to identify areas requiring repairs. Recognizing the critical importance of track inspection, this study introduces and integrates an intelligent automated system for analyzing the condition of train tracks. In summary, the study offers the following contributions:

• This study investigates the use of various machine learning and deep learning models for autonomously evaluating railway tracks, focusing on distinguishing between three distinct track conditions: wheel burn, superelevation, and standard track.

• A significant dataset is also produced for studies with the acoustic signals from an ECM-X7BMP microphone that was collected over one year.

• The Mel-frequency cepstrum coefficients (MFCC) and Constant-Q transform (CQT) characteristics of audio signals are combined with various classifiers to automatically detect track problems. Conditional GAN (CTGAN) is also used to create an equal amount of samples for each error.

The sections of this paper are grouped as follows: Section 2 provides a summary of several forms of fractures seen in railway tracks, as well as major studies on identifying defects in rail tracks. Section 3 describes data collecting procedures, equipment, and suggested study strategy. Section 4 presents the findings and comments, while Section 5 provides the conclusion.

Related work

Track inspection is an essential task that has been adopted periodically to control the conditions of rail tracks and avoid train accidents. Geometric inspection and structural inspection are two main classes for the inspections of rail tracks¹⁸. Structural checks are performed to detect structural faults such as wheel burn, superelevation, or other structural issues. Geometric inspections are used to identify geometric anomalies such as rail misalignment and other comparable degradation. Furthermore, geometric anomalies are caused by structural flaws, which can lead to train accidents. The authors explained various geometric and structural flaws in¹⁹.

Researchers worked on the detection of geometric defects with an SVM model in²⁰. The RAS problem-solving competition 2015 dataset was used for experimentation. The study considered some severe geometric defects which may increase the geometric defects. To detect structural defects, a structural inspection is performed using shallow machine learning methods in this study. SVM was used in this study which also worked on a novel parameter called positive and un-labeled learning performance (PULP). Moreover, PULP was applied to check the performance of models on different datasets comprising faulty results. In²¹, experimentation was performed to detect faults on railway tracks. Both Support SVM and CNN were utilized in this study for analyzing an image-based dataset. Rail fasteners are classified as missing, good, or broken. This technique showed improved accuracy in detecting defects in rail fasteners and ties.

The study²² investigated fault detection using traditional acoustic-based systems, enhancing performance and reducing train accidents through deep learning methodologies. Additionally, the research concentrated on LSTM, 2D convolutional, and 1D convolutional approaches. Various types of faults, such as wheel burn, superelevation, and normal tracks, were identified in this study. Experimental analysis was conducted on a real acoustic dataset to detect rail track faults.LSTM model shows improved results with 99.7% accuracy. In the study²³, local binary pattern (LLBP) was employed on railway images to classify track fasteners. Gabor filters²⁴, SVM²⁵, and edge detection²⁶ methods were utilized to identify fasteners in railway images. Faster Region-based CNN was utilized for detecting rail track faults in²⁷. CNN and ResNet-50 were applied in study²⁸ to detect structural defects and damages, particularly related to broken rail fasteners. The study utilized Haar-like feature sets, including geometric features for fasteners, achieving a 94% accuracy with CNN and 94.4% accuracy with ResNet-50. Additionally, various classification methods, including SVM, GNB, KNN, RF, Adaboost, and Gradient Boosting Decision Trees (GBDT), were tested and evaluated to detect and analyze missing clamps in the fastening structure in²⁹.

The categorization of railway cracks with acoustic-emission waves based on a multi-branch CNN is discussed in³⁰. The railway fastener defects are identified from images using CNN³¹, residual network³², GAN, faster region-CNN³³, and point cloud deep learning (PCDL)³⁴. Dynamic stiffness for rail pads was anticipated with machine learning techniques including KNN, multi-linear regression, regression tree, gradient boosting, RF, SVM, and MLP³⁵. Feature extraction approaches have also been investigated for rail track fault detection.

The study³⁶ introduced tree-based classification approaches such as RF and DT which performed a comparison of deep learning techniques for rail track inspections. The authors proposed a new RF-based approach that is used to combine LMD, TFD, and TD feature extraction for the detection of track slab deformation. In³⁷, an automated inspection technique based on IoT is presented for rail track fault detection. Acoustic data is used to rail track fault classification including wheel burn, crash sleeper, loose nuts, and bolts, low joint, creep, and point and crossing. The experimental results showed that acoustic data may successfully support selective track defects and localized these defects in real time. This method achieved a 98.4% accuracy with MLP.

Materials and methods

This section discusses dataset collection and strategy, feature extraction techniques, machine learning methods for classification, and the recommended approach.

Dataset collection

The dataset holds crucial importance in the automated identification of faulty railroad tracks. Illustrated in Figure 1, the mechanical cart provided by officials at the Rahim Yar Khan station of Pakistan Railways Khanpur district was utilized for collecting this dataset. A setup was arranged on-site at Khanpur’s train station to gather the necessary data. Positioned at a safe maximum distance of 1.75 inches from the point of contact between the wheel and the track, two microphones were installed. These microphones were affixed to the right and left sides of the cart for data collection purposes. The propulsion of the mechanical cart was facilitated by a generator, which operated at an average speed of 35 km/h to drive the cart’s engine.

Fig. 1

Mechanical railway cart used for data collection.

The audio data collection did not specify the geographic location. Two ECM-X7BMP Unidirectional electric condenser microphones, equipped with 3-pole locking small plugs, were mounted on the left and right wheels of the railway cart. These microphones possess an output impedance of 1.2 k and a sensitivity of 44.0 3 dB. Additional specifications of the microphones are detailed in Table 1.

Table 1 Important parameters of Sony ECM-X7BMP microphone.

Both microphones, positioned in separate locations, serve the purpose of recording. A single trigger button activates both microphones simultaneously to initiate data recording. The recorded data is stored as 16-bit audio files with the ”.wav” extension. The Sony ECM-X7BMP microphone is employed for this data collection process. As depicted in Figure 1, a metal strip is fashioned with one end serving as a secure mount for the microphone, while the other end is firmly screwed onto the cart. Foam or fur material is utilized to protect the microphone diaphragm from air currents. Without a windshield, wind or breathing can cause loud pops in the audio transmission.

Foam windshields are employed to mitigate cart vibrations, preventing their transmission to the microphone. Typically serving as the primary defense against wind noise, these windshields comprise open-cell foam covers surrounding the microphone. This design disperses and diminishes the acoustical energy of wind striking the microphone capsule, reducing low-end vibration. Streamlining these windshields is essential to ensure that wind flows around them rather than directly into them. Although some vibration remains uniformly present in the entire audio signal, it does not significantly impact the detection of faulty signals, as it is also present in normal track sounds. The windshields intercept air gusts before they can interact with the microphone diaphragm, effectively minimizing their impact. Using this setup, 720 audio recordings, totaling 17 seconds in duration, were captured during data collection at a sampling frequency of 22,050 Hz. Subsequently, the recordings were manually tagged to organize the dataset. They were then segmented into 758 frames using a window length of 1024 and 512 hops, based on the collected recordings.

Figure 2 presents the waveform and spectrogram of three different sample audio recordings: ‘wheel burn’, ‘superelevation’, and ‘standard’ track conditions. The waveform illustrates the amplitude variations over time, while the spectrogram provides a time-frequency representation, highlighting the intensity of different frequencies. The visual differences between these three track conditions are evident in both representations. The Mel spectrogram provides insights into the distribution of sound intensity across various frequency ranges. For instance, in the 64–256 Hz frequency range, the normal track sound exhibits an intensity level between approximately −30 dB and −60 dB. In contrast, the superelevation track demonstrates a higher intensity, ranging from −2 dB to −20 dB. Meanwhile, the track with wheel burn shows a broader variation in noise intensity, spanning from −20 dB to −72 dB within the same frequency range. These distinctions highlight the unique spectral characteristics associated with each track condition.

Fig. 2

Waveform and spectrogram representations of audio samples.

Proposed methodology for track fault detection

The proposed methodology for fault detection has been utilized in three scenarios. The architecture diagram of the proposed methodology is presented in Figure 3. In the first stage, features from the audio signal are extracted. In this study, we extract two types of features: one is MFCC while the other is CQT features. These feature sets are then divided into train and test parts.

Fig. 3

Architecture diagram based on feature extraction for faulty track detection.

The model is trained using a training dataset while evaluation is carried out using a testing dataset using accuracy, precision, recall, and F1 score. The acoustic features are then used to train machine learning and deep learning classifiers. The data size for each sample is 720 per class and 40 MFCC features are extracted from each sample. Along with MFCC features, another feature extraction technique i.e. CQT is also applied for models’ training.

Fig. 4

Architecture diagram using hybrid features for faulty track detection.

The architecture diagram for scenario 2 is shown in Figure 4. In order to enhance track fault detection accuracy, a feature fusion technique is applied where the features are combined. For that purpose, the best features are selected for model training. As a result, accuracy is improved for rail track fault detection.

Table 2 Class-wise data partitioning for model training and testing.

The data size is small to train machine learning algorithms, so, data are re-sampled by using CTGAN, and 400 samples are created for each class. These Samples are then split into train and tests with ratios of 0.8 to 0.2. The sample data distribution before and after the training and testing split is presented in Table 2. Machine learning and deep learning models are then trained by these training samples and then test samples are applied for prediction. The Architecture diagram for Scenario 3 has been explained in Figure 5.

Fig. 5

Architecture diagram based on hybrid features and newly generated features set for faulty track detection.

Obtaining implementation code

For reproducibility of the proposed approach, the implementation code has been made public on the GitHub, and can directly be accessed using the link https://github.com/Arehmans/railways.

Experiment details

Railway track inspection professionals from Pakistan Railways discovered and validated the damaged tracks. The cart was driven over the damaged rails, and the following tracks’ audio signals were recorded:

• Normal track (Unfaulty sound)

• Superelevation

• Wheel burn

The data is categorized as follows with respect to labels:

• Normal track - labeled as 0,

• Superelevation - labeled as 1,

• Wheel burn - labeled as 2

Wheel burns³⁸ occur when the driving wheels of locomotives skid along the rail surface, typically in areas with steep grades or after rainfall, due to insufficient hauling power to bear the train load, resulting in the rail surface melting. Superelevation³⁹ refers to the gradual elevation change between the rails of a railway track, creating banked turns that allow vehicles to navigate curves at higher speeds compared to level tracks, especially crucial on curved sections.

Both wheel burns and superelevation are recognized as critical factors contributing to railway derailments^40,41. Railways face various track issues such as cracked or broken rails, faulty welds, broad gauges, missing nuts and bolts, and disjointed tracks. However, this study concentrates specifically on wheel burns and superelevation, deferring other concerns for future research. The selection of the experimental route was deliberate; it was a heavily trafficked mainline, and during the study period, it was only affected by these two issues.

Mel-Frequency Cepstral Coefficients(MFCC)

MFCC features are discussed primarily for the detection of monosyllabic words within continuous speech rather than for speaker identification. The approach outlined in the paper aims to mimic the human ear’s functioning, leveraging the assumption that the human ear is a reliable recognizer of speakers. To capture the phonetically significant aspects of speech, frequency filters are organized linearly at lower frequencies and logarithmically at higher frequencies, forming the foundation of MFCC features.

Speech signals contain tones with varying frequencies, and the Mel scale is used to perceive pitch. Under the Mel scale, each tone corresponds to an actual frequency denoted as f (in Hz). Below 1000 Hz, the Mel frequency scale demonstrates linear frequency spacing, while beyond 1000 Hz, it adopts logarithmic frequency spacing. A reference point is set by a 1 kHz tone registering 40 dB above the perceptual hearing threshold, equivalent to 1000 Mel. In equation form, it can be written as

$$\begin{aligned} mel(f)=2592 \times log10(1+\frac{f}{700}) \end{aligned}$$

(1)

where mel(f) is the frequency in mels and f is the frequency in Hz. The final feature vector space F of size 40 is obtained as follows

$$\begin{aligned} F= \left[ \frac{1}{N}\sum _{i=1}^{758}a_{i1}, \frac{1}{N}\sum _{i=1}^{758}a_{i2}, \frac{1}{N}\sum _{i=1}^{758}a_{i3},...,\frac{1}{N}\sum _{i=1}^{758}a_{i40} \right] \end{aligned}$$

(2)

where i is the $i^{th}$ frame and N is the total number of frames i.e., 758.

Constant-Q-transform

Constant Q transformation is a technique to convert sound or signal data into frequency-domain data⁴². CQT mainly shows good performance for both perceptual and music processing. It is the same as Fourier transform (FT) whereas it has additional advantages. Firstly, it applies a logarithmic scale to ensure wide and narrow bandwidths in high-frequency and low-frequency regions. CQT is more useful than FT and reports low resolution in regions of low frequency. Additionally, the bandwidth is proportionally divided by the central frequency, making it simple to discriminate even if the frequency spans many octaves. Figure 6 provides a logarithmically spaced frequency resolution, offering a detailed representation of spectral content over time. The intensity variations indicate distinct spectral characteristics across different track conditions, with clear differences in frequency distribution and energy concentration. In the Wheel Burn case, higher intensity levels (−5 dB to 0 dB) appear as localized bright spots, indicating sudden energy spikes caused by irregular vibrations and impacts from wheel defects. This suggests a non-uniform frequency distribution, revealing abnormal disturbances in the track. In contrast, the Superelevation case exhibits moderate intensity (−10 dB to −20 dB) with a more evenly spread pattern, reflecting systematic frequency shifts due to track banking. This results in a smoother transition of forces acting on the track rather than abrupt variations. Lastly, the Normal Track serves as a baseline, showing lower intensity levels (−30 dB to −50 dB) with minimal bright spots, indicating uniform frequency distribution and the absence of major external disturbances. By analyzing these intensity variations, we can effectively diagnose track conditions, distinguishing between defects, structural features, and normal behavior.

Fig. 6

Constant-Q transformed spectra for different track conditions, (a) Wheel burn, (b) Superelevation, and (c) Normal track.

Constant Q transformation is used to analyze the frequency domain and can be estimated using the following equation⁴³.

$$\begin{aligned} y(w)=\frac{1}{n} \displaystyle \sum \limits y(b)j(b)e^(-ib/n) + \displaystyle \sum \limits y(b)j(b) \end{aligned}$$

(3)

where w is used for the sequence number of the spectral line and q is for the quality factor and its value is equivalent to the result for center frequency to the bandwidth. The center frequency is an exponential distribution. The window function n can be calculated as

$$\begin{aligned} n=\left[ q * \frac{gs}{g^4(w/a)}\right] \end{aligned}$$

(4)

where gs is used for sampling frequency, g is used for the low frequency of the musical signal, gw is the frequency value with spectral line and C is used for a number of spectral lines in an octave.

As an octave is separated by 12 semi-tones using an average temperament of 12, C mostly inputs a value for 12 or twelve multiple. However, CQT spectrum-frequency and scale-frequency have similar exponential distribution formula⁴³, CQT is used to analyze and process the musical signals. Therefore, the main issue of constant Q transformation is that the computation speed is very slow.

Composite travel generative adversarial network

Composite travel generative adversarial network is a GAN-based technique that is used for tabular data and sample rows from the distribution⁴⁴. The CTGAN approach is made up of 2 GAN networks including the sequence model and the tabular model. The main use for the tabular components is learning the joint distribution for elementary socio-demographic attributes. On the other hand, the sequential component is used to learn the distribution of trips that are selected by an individual per day. The properties include correlated features, different mixed data types like continuous or discrete features, problems in learning from higher sparse vectors, and potential mode failure due to the highest class imbalance. To solve these types of problems, we select composite travel generative adversarial network as the essential generative technique. CTGAN has many additional hyperparameters that are used to control its learning behavior and may impact the classifier performance for the computational time and quality of the generated data. CTGAN system considers that every component is employed as an independent network that is trained with its parameters based on data distribution. The presented method adopts just tabular components for the CTGAN approach and also trains the parameters. Conditional sampling only allows sampling using a conditional distribution with the CTGAN method, which means we may generate just values to satisfy the definite conditions. To overcome the multimodal and non-Gaussian distribution, the proposed method invents the mode-specific normalization in CTGAN.

Supervised machine learning models

The proposed approach uses several machine learning models to detect rail track defects. SVM, RF, RNN, DT, LR, NB, voting classifier, KNN, CNN, GRU, and LSTM are used in this study. These models are fine-tuned to optimize their performance.

Decision tree

The Decision Tree (DT) method, as discussed in⁴⁵, is a supervised classification technique characterized by its non-linear structure resembling a tree. In the context of evaluating faults and defects in transformers, the DT algorithm proves useful. In DT, connection points between branches represent conditions for differentiation, while the leaf nodes signify classifications. The classification process involves determining whether data meets the conditions outlined at each node, selecting appropriate branches to proceed, and repeating these steps until a leaf node is reached. In our study, DT is utilized with 2 hyperparameters. We specifically employ the ”max_depth” hyperparameter, set to 250, which restricts the decision tree’s growth to a maximum depth of 250 levels to prevent overfitting and manage complexity.

Support vector machine

SVM is a versatile linear model widely adopted for regression, classification, and various other tasks across numerous research articles^46,47. It operates by dividing sample data into distinct classes using a set of hyperplanes or a single hyperplane in a g-dimensional space, where g represents the number of features. SVM’s primary function is classification, aiming to identify the ”best fit” hyperplane that effectively separates different classes. In this study, we employ a ’linear’ kernel for the SVM classifier, which is commonly utilized when dealing with datasets featuring a high number of features. The SVM classifier offers two key advantages: high speed and enhanced performance even with a limited number of samples. For our current investigation, two hyperparameters are employed: a regularization parameter (C) set to 1.0 and the use of a ’linear’ kernel for experimentation purposes.

Random forest

RF is a classifier based on decision trees known for its ability to make accurate predictions by combining multiple weak learners, as described in⁴⁸. It employs the bagging method, where various types of decision trees are utilized during training, employing numerous bootstrap samples, as outlined in⁴⁹. These bootstrap samples are generated by randomly selecting subsets from the training dataset with replacement, maintaining a similar sample size as the original dataset. Ensemble classification is achieved by training multiple models and aggregating their results through a voting process. Several contributors have introduced ensemble learning methods, including boosting and bagging, which are widely utilized, as discussed in^50,51,52. Bagging, specifically, focuses on reducing variance in classification by training models on bootstrap samples. The definition of RF is as follows:

$$\begin{aligned} RF_p= & mode\{dt_1, dt_2, dt_3,...,dt_n\}\nonumber \\ \text {OR}\nonumber \\ RF_p= & mode\{\sum _{i=1}^Ndt_i\} \end{aligned}$$

(5)

where $dt_1, dt_2, dt_3,...,dt_n$ are the predictions by decision trees and $rf_p$ is prediction by RF using majority voting. We employed RF with three hyperparameters, as outlined in Table 3. The parameter n_estimators was set to 200, indicating that RF generated 200 decision trees for the prediction process. Additionally, max_depth was set to 50, limiting the depth of the decision trees to a maximum of 50 levels to prevent complexity and over-fitting.

Table 3 Hyperparameters used for machine and deep learning models.

Logistic regression

LR as referenced in⁵³, is a statistical classifier employed to address classification problems. When dealing with classification tasks where target variables are well-defined, logistic regression emerges as the primary choice. LR analyzes the association between one or more independent variables and categorical dependent variables by estimating probabilities using a logistic function. The logistic function, typically represented by a sigmoid curve, is defined as follows:

$$\begin{aligned} Y=\frac{h}{1+E^{-n(u-u_0)} } \end{aligned}$$

(6)

where E is the classification⁵³, Euler number u0 is used for the value of the sigmoid mid-point, h is the maximum value of the curve, and n is used for the steepness of the curve. LR performs better on binary classification and demonstrates improved performance for the text.

Naive Bayes

NB classifier is a supervised learning technique that is based on the Bayes formula and used to solve classification issues⁵⁴. NB is essentially applied to detect faults in the rail track. NB is one of the easiest and most effective classifiers that helps to build machine learning-based methods that can predict quickly. NB is a probabilistic model which means it performs prediction based on probability for an object. Suppose a set of d vectors $E = {e1,e2,...,ed}$ and performs classification along a set F of p classes $F = {f1,f2,...,fp}$, Bayesian models estimate the probabilities for every category Fi given an Ej is described in the below equation⁵⁴

$$\begin{aligned} P(e_i|f_k )= \frac{P(f_k |e_i)P(e_i)}{(P(f_k))f(x)} \end{aligned}$$

(7)

where $P(e_i)$ is the probability and picked randomly has vector $e_i$ as its demonstration that belongs to $f_k$. To estimate the $P(f_k|e_i)$, NB considers that the probability for a given value is independent. NB shows improved results than other classifiers. Additionally, only values are used as the predictors, the simplification of naive allows computing the model for data that is associated with this technique. It is possible to define $P(f_k|e_i)$ as the product for the probabilities of every term that appears using this simplification. However, $P(f_k|e_i)$ is estimated using equation⁵⁴.

$$\begin{aligned} P(G_k|f_i)=\Pi _1^t (P(U_jk |f_i)) \end{aligned}$$

(8)

K Nearest Neighbors

KNN classifier stands out as the most straightforward and non-parametric supervised machine learning technique, utilized for regression, classification, and addressing missing value imputation problems⁵⁴. Its approach involves storing all available data and determining the classification of a data point based on similarity. During the training phase, the KNN algorithm solely retains the dataset and assigns the data to a category highly resembling the new data. To precisely define the nearest neighbors, a distance metric such as Manhattan or Euclidean distance is computed⁵⁴. KNN is alternatively known as a lazy or instance-based learner. However, it’s worth noting that KNN cannot predict values that fall outside the range of the sampled data.

Ensemble classifier

Ensemble voting is a voting classifier that combines several classifiers into a single model which is more robust than individual models⁵⁵. For the current study, hard voting is used. Every model votes for a category in the hard voting and the category with the maximum votes wins. Every model in soft-voting allocates a probability value to every data point that belongs to a specific target category. In the presented model, we combine LR, GNB, and SVC classifiers into a single method.

Deep learning models

Besides using machine learning models, several deep learning models are used.

Long short-term memory

LSTM as referenced in⁵⁶, resembles RNN (Recurrent Neural Network) but incorporates efficient memory cells designed to either forget or retain information. It addresses the problem of long-term dependency by employing a chain of RNN modules. The LSTM architecture includes four gates: the update gate, output gate, forget gate, and input gate. The forget gate determines whether the information is discarded from the cell state, while the input gate, consisting of a tanh layer and a sigmoid layer, determines which values will be modified. The update gate refreshes the old cell state with the value derived from the input gate. Finally, the output gate is utilized to determine the value to be outputted from the layer⁵⁶.

$$\begin{aligned} R_L=\phi (Y_R[G_L-1,Z_L]+A_R) \end{aligned}$$

(9)

where $G\_L$ denotes the weight matrix and $A\_R$ is used for the bias vector. Suppose $R\_L$ is a number between 0 and 1, then 0 indicates that the value is to forget and 1 indicates to keep the value.

$$\begin{aligned} P_L= & \phi (Y_P[G_L-1,Z_L]+A_L) \end{aligned}$$

(10)

$$\begin{aligned} W^ \sim _L= & tanh(Y_W[G_L-1,Z]+A_W) \end{aligned}$$

(11)

where $Y\_P$ and $Y\_w$ are used for weight matrices and $A\_L$ and $A\_W$ are used for bias vectors. For output, $P\_L$ and $W\_L$ are used.

$$\begin{aligned} W_L=R_L* W_L-1 + P_L* W^ \sim _L \end{aligned}$$

(12)

where $R\_L$ is used to decide which information is to be forgotten. $P\_L * W\_L$ chooses the total number of values that are used to modify the cell.

$$\begin{aligned} P_E= & \phi (Y_E[G_L-1,Z_L]+A_E) \end{aligned}$$

(13)

$$\begin{aligned} H_L= & E_L * tanh(f_l) \end{aligned}$$

(14)

where PE is used to decide which is the output state. The new cell state WL is multiplied by EL. The tanh function is used to achieve HL which is the output of PE. The presented model used LSTM which takes the least time for training. LSTM is the most efficient method than other machine learning techniques.

Convolutional neural network

CNN is mostly used to deal with the variability of 2D shapes⁵⁷. This architecture is tested for feature extraction of images. CNN contains two layers including the convolution layer and the pooling layer. The convolutional layer is used to perform convolution of the previous layer using the sliding filter to attain the output feature map, where $F_c^{de}$ denotes the $p^{thoutput}$ feature map in de layer, $j_l^{(de-1)}$ is used for $l^{th}$ input feature map in $(de-1)$ layer. $\phi$ is used for the sigmoid function that is employed as the network’s activation function. Both $Y_{pl}^{ae}$ and $P_p^{ae}$ are used for filters that create the training parameters of convolutional layers as in bellow equation⁵⁷

$$\begin{aligned} F_c^{de} =\phi (\displaystyle \sum \limits _{l=1}^k j^{de-1} * Y_pl^{ae} + P_p^{ae}) \end{aligned}$$

(15)

To minimize the feature-map resolution and sensitivity for output, the pooling layer is used. The max pooling is commonly used for pooling in CNN. The max pooling is described⁵⁸ as in the bellow equation.

$$\begin{aligned} H_p^{di} =max\lbrace {b_1,b_2,b_3,b_4}\rbrace \end{aligned}$$

(16)

where $H_p^{di}$ shows the $p^{th}$ output feature map of pooling layer.

Recurrent neural network

RNN model⁵⁹ is used to save the output for specific layers and feedback to the input in sequential form to predict the output. RNN is considered to handle the sequential data.

RNNs memorize the previous inputs due to internal memory. It simulates a discrete-time dynamical system that has xe for the input layer, ye for the hidden layer, and ze for the output layer, and e is used to denote time. The dynamical model is defined as in equation and bellow equation⁵⁹

$$\begin{aligned} y_e= & G_y(x_e,y_{e-1}) \end{aligned}$$

(17)

$$\begin{aligned} z_0= & G_0(y_e) \end{aligned}$$

(18)

where $G_y$ and $G_0$ are functions that are used for state transition and output respectively. Each function is parameterized by a set of parameters as $\theta y$ and $\theta 0$.

Gated recurrent unit

GRU represents the next evolution of RNNs, utilizing the hidden state for information transmission. Unlike LSTM, GRU incorporates only two gates: an update gate and a reset gate⁶⁰. The update gate functions similarly to the input and forget gates in LSTM, determining which information to discard and what new information to incorporate. On the other hand, the reset gate is responsible for determining the extent to which information should be forgotten.

Results and discussion

The experiments are conducted using the Google Colab service alongside a Python Jupyter Notebook. Librosa is employed to extract MFCC and CQT features, while machine learning models utilize the sci-kit-learn package and deep learning models utilize the TensorFlow library.

An equal number of data points for each class are used for experiments. The dataset was structured to ensure uniform representation across all categories including normal track (0), superelevation (1), and wheel burn (2), to prevent class imbalance and ensure fair model training and evaluation. Performance evaluation of the classifiers is carried out using standard parameters such as accuracy, precision, recall, and F1 score, which are computed using the following equations.

$$\begin{aligned} Accuracy= & \frac{T_rP_o+T_rN_e}{T_rP_o+T_rN_e+ FP_o+FN_e} \end{aligned}$$

(19)

$$\begin{aligned} P= & \frac{T_rP_o}{T_rP_o+ FP_o} \end{aligned}$$

(20)

$$\begin{aligned} R= & \frac{T_rP_o}{T_rP_o+ FN_e} \end{aligned}$$

(21)

$$\begin{aligned} F1-score =2 * \frac{P*R}{P+R} \end{aligned}$$

(22)

Results of machine learning classifiers using MFCC features

Table 4 depicts the accuracy results of different machine learning models using MFCC features. Several models have been applied for experiments including DT, SVM, KNN, LR, NB, RF, and voting classifiers. Accuracy results for DT, SVM, KNN, LR, and NB are 96%, 99%, 85%, 97%, 78%, and 97%, respectively. Experimental results show that tree-based models like DT and RF perform best with accuracy results of 96% and 99%, respectively. While regression and probabilistic-based models perform poorly like NB and KNN yield 78% and 85%, respectively. The accuracy of the voting classifier with hard and soft voting is 98%.

Table 4 Results of machine learning classifiers using MFCC features.

Table 5 Results for CQT features using machine learning classifiers.

Results of machine learning classifiers using CQT features

Table 5 shows the performance of different machine learning models using CQT features. Accuracy scores for DT, SVM, KNN, LR, NB, and RF are 96%, 99%, 85%, 97%, 78%, and 97%, respectively. Results demonstrate that tree-based models like DT, RF, and liner based models perform best with accuracy scores of 95%, 97%, and 93% as compared to regression and probabilistic-based models like NB and KNN with 87% and 72% accuracy scores, respectively. The liner-based algorithm also performs well with an accuracy of 93%. Accuracy of the voting classifier with hard and soft voting yield 95% and 94% accuracy, respectively.

Results of classifiers using hybrid features

Feature fusion is a technique that is formulated with multiple features that are extracted from the same dataset. The benefit of feature fusion is the increased versatility in feature sets; different types of features can be extracted and used for model training. In Table 6 accuracy of different machine learning models is reported using a fusion of MFCC and CQT. Classifiers like DT, SVC, KNN, LR, NB, and RF yield accuracy scores of 93%, 95%, 85%, 93%, 47%, and 97%, respectively. Probability-based models yield less accuracy for all scenarios.

Table 6 Results of machine learning classifiers using hybrid features with CTGAN.

Results of machine learning classifiers using MFCC features with CTGAN

Data augmentation methods like GAN create new data samples. GAN creates distinctive samples that imitate the feature distribution of the original dataset using random noise taken from latent space. Table 7 displays the accuracy results of classifiers using MFCC features along with CTGAN. As CTGAN techniques create more sample features with the big size of the dataset models can be better tuned and results are improved substantially. SVC, KNN, LR, NB, RF, and voting classifiers with hard and soft voting yield 100% accuracy after applying CTGAN while the performance of DT is decreased to 94%.

Table 7 Results of models using MFCC with CTGAN.

Results of machine learning classifiers using CQT features with CTGAN

In Table 8, the performance of machine learning classifiers with CQT features along with the augmentation technique CTGAN is evaluated. LR, NB, RF, SVC, and voting classifier yield accuracy results of 100% while DT and KNN perform less with 92% and 71% accuracy, respectively. Overall, the performance of probabilistic classifiers is increased.

Table 8 Results of machine learning classifiers using CQT features with CTGAN.

Results of machine learning classifiers using Hybrid features with CTGAN

We have performed experiments along with a combination of both features MFCC and CQT after data augmentation using CTGAN and results are given in Table 9. Different machine learning classifiers are trained and accuracy results for SVM, KNN, LR, NB, and voting classifiers indicate a 100% accuracy. DT shows poor performance with an 85% accuracy because it is a Singleton algorithm and when data size increases the complexity level is also increased resulting in a decrease in its performance.

Table 9 Results of models using MFCC and CQT features from CTGAN data.

Results of deep learning classifiers using MFCC and CQT features

Table 10 depicts the accuracy results of different deep learning models using MFCC and CQT features extracted from the audio signal with an 80 to 20 train test ratio.

Table 10 Results of deep learning classifiers using hybrid features.

Deep learning models have been applied in this experiment including LSTM, CNN, RNN, and GRU. Results show that LSTM, CNN, RNN, and GRU models show the accuracy of 33%, 93%, 72%, and 89%, respectively. While with CQT features deep learning classifiers yield 36%, 89%, 63%, and 88%, respectively. The overall performance of deep learning classifiers is lower as compared to machine learning classifiers because of dataset size. Deep learning models perform best on large datasets.

Table 11 Results of deep learning classifiers using hybrid features with CTGAN.

Results of deep learning classifiers using MFCC and CQT with CTGAN

Table 11 shows the results for deep learning models after data augmentation is performed using CTGAN and MFCC and CQT features are used for model training. For MFCC features with augmentation, LSTM, CNN, RNN, and GRU yield 48%, 100%, 76%, and 100% accuracy, respectively. From the results, it can be observed that the models’ performance is enhanced as the dataset size is increased. Similarly with CQT features with augmentation, accuracy for LSTM, CNN, RNN, and GRU is 40%, 100%, 51%, and 100%, respectively. The accuracy is also improved in this case.

Results of deep learning classifiers using hybrid features and hybrid features with CTGAN

Table 12 shows results for two types of experiments; in the first part, a fusion of both MFCC and CQT is used to train deep learning classifiers while the second part involves experiments with feature fusion from CTGAN-generated data. For the first scenario, the accuracy results for LSTM, CNN, RNN, and GRU are 36%, 89%, 63%, and 88% respectively. Results show a slight improvement as compared to the single-feature extraction technique. On the other hand, results for the second part are much better with 40%, 100%, 78%, and 100% accuracy for LSTM, CNN, RNN, and GRU, respectively. Figure 7 shows the accuracy scores comparison between all approaches.

Table 12 Results of deep learning models for hybrid MFCC+CQT with CTGAN.

Fig. 7

Comparison of accuracy scores: (a) Results of machine learning classifiers with MFCC, CQT, and hybrid features, (b) Results of deep learning classifiers with MFCC, CQT, and hybrid features, and (c) Results of machine learning classifiers with MFCC, CQT, and hybrid features using GAN data, and (d) Results of deep learning classifiers with MFCC, CQT, and hybrid features using GAN data.

K-fold cross-validation results

We have also performed k-fold cross-validation to check the performance of the model that is outperformed and gives a 1.00 mean accuracy score with +/−0.00 standard deviation using the proposed approach. The results of our approach using 10-fold cross-validation are the same as per the train test split method. The results of K fold cross-validation with and without CTGAN are shown in Tables 13 and 14. After applying CTGEN with feature extraction techniques the machine learning models improve the accuracy which shows that CTGAN helps to generate enough data for the learning models.

Table 13 K-fold cross-validation results with MFCC, CQT features using machine learning models.

Table 14 K-fold cross-validation results with MFCC, CQT features, and CTGAN augmentation using machine learning models.

Comparison With existing studies

Several studies have worked on the detection of rail faults using machine learning approaches; some of these studies used the same dataset. For the studies which used the same dataset we compared their results while for those which used other datasets, we deployed their approaches on the currently used dataset and performed experiments for a fair comparison. The study⁶¹ performed experiments on the same dataset while the study^37,51,57 carried out experiments on other datasets for railway track fault detection. So we deploy the proposed approaches in^37,51,57 using the currently used dataset and show the comparative performance in Table 15. The results show the significance of the proposed approach indicating the superior performance of the proposed approach. This study focuses on dataset size which is ignored by previous studies which elevated the performance of the machine learning models. In addition, we also deployed hybrid features while most of the existing studies only used MFCC features. The use of hybrid features helps to achieve better results than merely using MFCC features.

Table 15 Comparison with other existing studies.

Conclusions and future work

The railway network serves as the backbone of today’s transportation system and its regular operations are very important for the transportation of goods and humans. Cracks, ballast issues, burn wheels, superelevation, etc. can disrupt railway tracks and cause financial and human losses. Automatic detection of such faults can avoid laborious and error-prone manual fault detection. Contrary to existing studies that rely on MFCC features, this study proposes the use of hybrid features including MFCC and CQT features with an enlarged audio dataset and shows improved performance with an ensemble model. In addition, using the CTGAN model for generating additional samples yields better performance than existing state-of-the-art approaches for railway track fault detection. An accuracy of 100% can be obtained using CTGAN and hybrid features.