Introduction

Classifying objects by acquiring and examining their primary distinguishing characteristics is the foundation of the recognition process, which is a high-level computer vision computing task1. Certain physical characteristics, such as color, form, and roughness themes, have been utilized to differentiate between different flowers. Finding appropriate hue, form, and feature descriptors is challenging, and making the classifier capable of choosing important features is equally challenging. For the purpose of creating a robotic apple pollination system, identifying specific king flowers within flower clusters is essential. During the synthesis period, apple flower clusters open successively from the king flower to the lateral flowers, providing a chance for selective pollination2. During the synthesis period, apple flower clusters open successively from the king flower to the lateral flowers, providing a chance for selective pollination. As a result, keeping an eye on the flower’s blossoming stage is essential to precisely identifying the pollination targets and time3. Utilizing ANN in conjunction with computational image analysis methods, the flower recognition system was created. In order to recognize different species of flowers, several classification algorithms have been developed. These include genetic algorithms, artificial neural networks, and other machine learning algorithms4. Flowers belong to a broad class of plant life, many of which have relatively similar characteristics and appearances; however certain flower species differ from one another. Because of these similarities and differences, identifying flowers with a high degree of accuracy is an extremely difficult task. Identifying flowers from their images using standard methods such as visiting websites on the Internet, searching for keywords on search engines, or consulting floral reference books was ineffective5.

Current automated computer vision systems for flower identification rely on manually designed methods that perform inadequately and only function in certain situations. An automated method for identifying flowers may be used with a variety of flower species and is resistant to uncontrolled settings6. To produce apples of a greater quality and quantity, management practices that accommodate the physiological condition of individual trees must be implemented7. A versatile method for segmenting images is employed, incorporating two feature learning methods such as CNN and MLP networks8. In particular, the DL network’s output gives the active contour model a shape restriction9. Given the diversity of the species and the potential for visual similarity between many flower species, flower classification is a difficult undertaking. The process of classifying flowers entails a number of challenges, such as photos with poor quality and noise; images obscured by plant stems and foliage, and occasionally even by insects10. Computerized living plant recognition focuses on extracting stable features from plants. Consequently, evidence found in leaf veins is crucial for recognizing living plants. Neural networks with deep layers are frequently utilized in computer vision algorithms due to their increased capacity for recognizing patterns in images. Deep learning is a potent artificial intelligence approach that is becoming a common method of text and image recognition and categorization in modern times. The deep learning technology can be used to extend its application towards flower species recognition. Deep learning technology is essential for expanding its use in interdisciplinary fields to include diagnostics, forecasting, and decision-making. Since deep learning technology has improved to automate digital agriculture, there is a possibility that malicious users would damage the deep learning model that has been produced and attempt to breach model confidentiality through security vulnerabilities11,12,13,14,15,16,17. The input dataset had the biggest impact on the potential to alter the DL model’s prediction. Therefore, the suggested model, SRB-VGG19, took into consideration and addressed the adversarial attempt to alter the training dataset samples in the form of a data poisoning FGSM attack on the DL model.

Paper organization

This article was organized as follows. Section “Motivation and objectives of this research work” portrays the contributions of this research work. Section “Background study” summarizes the methodology and the findings of the background study articles towards the flower species recognition. Section “Research methodology of proposed SRB-VGG19” provides the research methodology of proposed SRB-VGG19. Section “Mathematical modelling of SRB-VGG19” explores the mathematical modeling of proposed SRB-VGG19. The implementation results of the proposed SRB-VGG19 are discussed in Section “SRB-VGG19 implementation results and discussions”. Finally, the proposed SRB-VGG19 model was concluded in the Section “Security countermeasure FGSM attack implementation results and discussion” with the findings and future enhancements.

Motivation and objectives of this research work

Classifying flowers can be challenging because there is a lot of variation among flower species and some may look similar to one another. Effective flower species classification requires a suitable classification method that makes use of deep learning technologies. This motivates to design the proposed SRB-VGG19, which is highly effective at classifying flower species and is inspired by VGG19 model. By capturing high-level representations of the data, dense layered block is incorporated into VGG19 to improve feature extraction and increase the model’s ability to perform accurate classification. This motivates the design of proposed SRB-VGG19 Dense ANL model that can deliver consistent performance in noisy, distorted input data in addition to achieving high accuracy on clean data. The motivation of this research is to close the gap between the latest deep learning architectures and the growing demand for reliable AI systems that can function consistently in challenging real-world situations. By using adversarial training, this work provides a balance between preserving the model’s robustness against adversarial attacks while preserving high performance on clean data. The proposed SRB-VGG19 Dense ANL model offers an innovative approach for boosting deep convolutional networks’ resilience and versatility, opening the door for more dependable AI systems in real-world settings. The main objective of this work is to adapt Sobel filter and the Restricted Boltzmann Machine approach through unsupervised learning for the purpose of feature extraction. The second objective is to finetune the VGG19 model by integrated with the dense layered block. The third objective is to address the issue of data poisoning attack through Fast Gradient Sign Method applied to the input data samples18,19.

Contribution of this research work

The following are the main contributions of this research are four folded.

  1. (i)

    The first contribution deals with the dataset preparation by means of feature extraction through the use of the Sobel filter and RBM neural network approach through unsupervised learning to form the RBM Component Vectorized (RBMCV) flower images.

  2. (ii)

    The second contribution is the design of four layered dense block for embedding with the SRB-VGG19 Dense Model

  3. (iii)

    The third contribution is the fine-tuning of VGG19 and DenseNet to design the proposed SRB-VGG19 with the following sub models. The RBMCV flower images are processed with five convolutional blocks to form RBMCV flower feature maps.

    1. a.

      The SRB-VGG19 FCL model retrieves the RBMCV flower feature maps and processed with two FCL layers along with an third additional FCL layer to generate optimized RBMCV feature map. The third updated Dense Layer block consists of one Batch Normalization, one ReLU, and one dropout layer. Finally the SRB-VGG19 FCL model ends with the softmax layer with five neurons to classify the five flower species.

    2. b.

      The SRB-VGG19 Dense model retrieves the RBMCV flower feature maps and processed with two Dense block to generate optimized RBMCV feature map. The addition of four layered Dense block in this model as shown in the Fig. 1 overcomes the problem of vanishing gradient. The dense blocks are composed of four convolution blocks connected in series. The single convolutional block in the dense block consists of batch normalization, RELU and convolution that allow cross-layer connections between two non-adjacent convolution blocks as shown in Fig. 1. Each input convolutional block contains feature maps from all the previous convolutional blocks. Finally, the SRB-VGG19 Dense model ends with the softmax layer with five neurons to classify the five flower species.

      1. (iv)

        The fourth contribution attempts to overcome the data poisoning FGSM attacks that are vulnerable to proposed SRB-VGG19 to design SRB-VGG19 Dense ANL Model. Inspired by the ANL against FGSM attack, the ANL layer was created as the hidden layer in between the Convolutional layers of the four layered dense block. The ANL layer was designed inside the hidden layer of the dense block as shown in the Fig. 2.

Fig.1
figure 1

Design of four layered dense block.

Full size image
Fig. 2
figure 2

Design of ANL layer to form dense ANL block.

Full size image

Background study

Roses are vulnerable to severe temperature swings, damage from drought, and insufficient water. Due to the benefits of a controlled environment, this led to a growth in greenhouse production in order to satisfy expanding demand with the best possible supply. The dataset used in rose greenhouse cultivation was subjected to ML models. It categorizes the best greenhouse conditions for improving the condition of roses and achieving maximum rose production1. As a result, keeping an eye on the flower’s blossoming stage is essential to precisely identifying the pollination targets and time. A machine vision system was created in order to gather pictures of two different apple types in an orchard setting. A machine vision system was created in order to gather pictures of two different apple types in an orchard setting. In order to recognize and find the most prominent flowers from an apple flower dataset during the blooming stage, from initial king bloom to full bloom, a Mask R-CNN-based recognition system and a king flower segmentation technique were built2. Beginning with image improvement, dataset images are cropped to generate a more acceptable dataset. Subsequently, image segmentation was employed to distinguish the foreground from the background, using active contour. All three features are color, texture, and shape and they were extracted, with GLCM serving as the texture identifier and Invariant Moments as the form descriptor3.

Flowers belong to a broad class of plant life, many of which have relatively similar characteristics and appearances; however certain flower species differ from one another. Because of these similarities and differences, identifying flowers with a high degree of accuracy is an extremely difficult task. Identifying flowers from their images using standard methods such as visiting websites on the Internet, searching for keywords on search engines, or consulting floral reference books was ineffective4. With Atrous convolution, it is possible to regulate the Deep CNN feature response computation resolution. Additionally, it enables us to efficiently expand the filter’s field of view to include more contexts without adding more parameters or computing power. Objects of different scales are pooled in an atrous spatial pyramid5. By adding a CNN, the need for manually constructed feature extractors of the model is avoided, and the optimal accuracy is attained by strengthening the model. Finally, to reveal pertinent vein configurations by using a straightforward visualization technique was done to analyze the produced models6. Apple blossoming abundance was detected in a high-density apple orchard by analyzing HSL (hue, saturation, and luminance) photographs to determine the number of flower clusters (FC) of individual trees7.

By adding contextual information regarding the image data’s collection method, which correlates with some of the observed appearance changes and class distributions in the data, the MLP and CNN networks were expanded. The watershed segmentation (WS) and circular Hough transform (CHT) algorithms are used to process the pixel-wise fruit segmentation output in order to identify and count individual apple fruits8. The apple, peach, and pear flower datasets are utilized to fine-tune the semantic segmentation network DeepLab-ResNet, which is then used for detection. An active contour model is utilized to improve the network’s coarse segmentation results, presuming that the network can identify the flower object approximately and that the flower’s color differs from the backdrop9. Neural Architecture Search –Feature Pyramid Network (NAS-FPN) along with Faster-RCNN deployed with transfer learning process was adopted to perform the flower classification process20. The minimal Redundancy Maximum Relevance (mRMR) approach was utilized in the Deep CNN feature selection algorithm to choose the more effective flower features. Using the collected features, a SVMclassifier with a Radial Bases Function (RBF) kernel was used to categorize the floral species21. For instance, a set of aesthetic characteristics was taken from a given set of flower images and used to generalize these images to new images with potentially unknown flowers, rather than training for a single category of flowers based solely on human developed features like SIFT and HoG. The classification system known as sparse representations is utilized to forecast the characteristics of a certain floral image from any category. In order to perform better during the flower classification stage, the most discriminative qualities were identified among others using the genetic algorithm22. In the Blockchain Data Lake framework, the original image data was verified for accuracy and uniqueness using the Color Consistency Automatic Network (CCAN) based flower classification method23. Pattern similarities and variances are distinguished using Multilayer NN and the pairwise confusion loss function based on pair similarity. The fine grain size serves the objective function of enabling the various flower patterns24. The Single Short Detector (SSD) and Faster-Recurrent CNN (Faster-RCNN) along with transfer learning model are used to locate, identify, and categorize floral items25. An image-based identification tool that allowed users to query or add to the framework with additional information was made accessible as a mobile and internet-based application that synchronizes with the expanding data. This makes it possible to query the system year-round and to create a plant observation with complementing images26. Multimodal CNN represent each word in a text as a vector and extract visual information. Then, using a CNN model for text data, the textual features are retrieved27. Due to a multitude of plant species variations, inaccurate image pre-processing methods like edge detection and contour extraction, and a lack of suitable models or representation strategies, computer-aided plant authentication remains most difficult tasks in CV28. The flowers were categorized using textural characteristics, such as the Gray level co-occurrence matrix (GLCM). The flower classification was done using multidimensional genetic association rule mining29.

Apple trees need to have some of their blossoms and fruitlets cut early in the period of development in order to maximize fruit production. The amount of blossoms in the orchard, or the bloom intensity, indicates how much needs to be removed30. Few labeled flower data are needed for multi-class fruit blossom detection in the Deep CNN framework, which increases the training process by lowering the sample size and broadens the model’s relevance to a larger variety of flower categories. Location guidance module (LGM) was utilized to emphasize foreground regions in query images that resembled the bloom references given by support images in order to increase the likelihood of flower31. The Ghost module was used to replace the Convolution module in the Neck section of YOLOv5’s network in order to detect apple blossoms. ShuffleNetv2 used Channel Splitting technique to lower the memory access cost. Ghost module kept the same detection performance while lowering the volume layer’s processing cost32. In order to create a consistent environment for accurate luminescence and color assessment even in the presence of fluctuating light enlightenment the color and brightness compensation approach employed a scanning method. Adaptive camera parameter modification was calibrated based on the relationships between color discrepancies and camera attributes in order to produce the optimum color for tomato flower recognition33. Table 1 shows the inferences from the literature review.

Table 1 Inferences from the literature review.
Full size table

From the initial king flower to the complete flowering condition, a king flower segmentation algorithm and a Mask R-CNN-based detection model were created for identifying and locating the king flowers in an apple blossom dataset34. Contrast Limited Adaptive Histogram Equalization (CLAHE) and filtering approaches are used to pre-process the ensemble deep learning-based flowers categorization model. During the optimal pattern extraction step, which takes into account the pre-processed images, the best hybrid patterns are extracted from the Local Binary Pattern (LBP) and Local Vector Pattern (LVP).35. The Xception CNN incorporates both the channel attention and spatial attention mechanisms. The network was refined by combining Triplet Loss and Softmax Loss in the network loss layer, resulting in feature embedding space for flower classification36. The limitation and the advantages of the literature work is shown in Table 2.

Table 2 Advantages and Limitations of Existing Work.
Full size table

Research methodology of proposed SRB-VGG19

The overall research methodology of the proposed SRB-VGG19 is shown in Fig. 3. The proposed SRB-VGG19 model Overall Framework was depicted in Fig. 4. The stage 1 of the proposed SRB-VGG19 model initiates by collecting the images from Flowers recognition Dataset. Stage 2 deals with the formation of Sobel filtered RBM Component Vectorized flower images. The dataset pre-processing for the flower image was done by forming the Sobel filtered image by performing spatial gradient measurement on the image and includes only the high spatial frequency edges from the images.

Fig. 3
figure 3

Research methodology of SRB-VGG19.

Full size image
Fig. 4
figure 4

Overall framework of SRB-VGG19.

Full size image

The sobel filtered image contains only the high intensity pixels that corresponds to the image edges and its module is shown in Fig. 5. This sobel filtered image are fitted to RBM to create RBM component vectorized images. The RBMCV flower image contains only the essential edges of the images that helps for identification of the flower image. Stage 3 focuses on selecting the best CNN towards classifying the flower species. Stage 4 proposes the design of SRB-VGG19 model by replacing FCL of existing VGG19 with the FCL and dense block. The sobel filter of the image was formed by retrieving the pixel values of the image. Then the horizontal gradient and vertical gradient of the image was found to compute the final gradient. The horizontal gradient of the image was created by finding the difference between the two pixels in the column of the image. The vertical gradient of the image was created by finding the difference between the two pixels in the rows of the image.

Fig. 5
figure 5

Sobel filter module of SRB-VGG19.

Full size image

This sobel filtered image are fitted to RBM that provides unsupervised ANN which creates the RBM component vectorized images using the probabilistic values of the pixels in the sobel filtered image and its module is shown in the Fig. 6. To determine which best CNN model, the training images from the flower recognition dataset are fitted using the current models, including DenseNet, AlexNet, LeNet, VGG19, ResNet, Inception, and Xception. According to experiment results, VGG19 and DenseNet can classify floral species with an accuracy of above 80%. So, the VGG19 and DenseNet were selected to fine tune its structure to design the proposed SRB-VGG19 FCL and SRB-VGG19 Dense model.

Fig. 6
figure 6

RBM component module of SRB-VGG19.

Full size image

SRB-VGG-19 FCL model architecture

The proposed SRB-VGG19 FCL model initially begins by using the Sobel filter and RBM to turn the input image into an RBM Component Vector of Flowers that only includes the essential flower edges. After that, RBMCV flower feature maps are produced by processing the RBM Component Vector of Flower images through five convolutional blocks. In order to create an optimized RBMCV feature map, the acquired important feature maps are then processed using a FCL made up of three dense layers. The third updated Dense Layer block, which consists of one Batch Normalization, one ReLU, and one dropout layer, is then fitted to the optimized feature map. The comparison of the VGG19 in SRB-VGG19 FCL model is shown in Fig. 7.

Fig. 7
figure 7

Comparison VGG19 in SRB-VGG19 FCL model.

Full size image

The SoftMax layer comes after the FCL. After that, the optimized RBMCV feature map is used for batch normalization, which centers the aligned features in the active region and removes the unaligned optimized RBMCV feature map weights from focus. Subsequently, nonlinearity is added to the aligned features using the ReLU activation function, assisting the feature vector in learning its complex form. The erroneous noise in the aligned weights of the optimized RBMCV feature map is then removed using dropout and the SRB-VGG19 FCL model architecture is shown in Fig. 8.

Fig. 8
figure 8

Architecture of SRB-VGG19 FCL model.

Full size image

SRB-VGG-19 dense model architecture

The proposed SRB-VGG19 Dense model initially begins by using the Sobel filter and RBM to turn the input image into an RBM Component Vector of Flowers that only includes the essential flower edges. The RBMCV flower images are processed with five convolutional blocks to form RBMCV flower feature maps. The comparison of the VGG19 in SRB-VGG19 Dense model is shown in Fig. 9.

Fig. 9
figure 9

Comparison VGG19 in SRB-VGG19 dense model.

Full size image

The SRB-VGG19 Dense model retrieves the RBMCV flower feature maps and processed with two Dense block to generate optimized RBMCV feature map. The dense blocks are composed of four convolution blocks connected in series. The single convolutional block in the dense block consists of batch normalization, RELU and convolution that allow cross-layer connections between two non-adjacent convolution blocks. Each input convolutional block contains feature maps from all the previous convolutional blocks. Finally, the SRB-VGG19 Dense model ends with the softmax layer with five neurons to classify the five flower species. After that, feature maps are produced by processing the RBM Component Vector of Flower images through five convolutional blocks. The SRB-VGG19 Dense model architecture is shown in Fig. 10.

Fig. 10
figure 10

Architecture of SRB-VGG19 dense model.

Full size image

SRB-VGG-19 dense ANL model architecture

The ANL Layer was added to the four layered dense block of SRB-VGG19 Dense model and the Dense ANL Block was shown in Fig. 11.

Fig. 11
figure 11

ANL layer design of dense block in SRB-VGG19 dense model.

Full size image

When the data poisoning attack takes place by adding the adversarial noise data sample as the input to confuse the trained DL model, there is a possibility of performance degradation in the system. This leads to assume the fake noise data samples also to be the real samples and the training happens along with the noise data ending with successful data poisoning attack. To overcome this, Adversarial Noise Layer (ANL) can be appended that are crafted with noises in the intermediate layer while designing the CNN layer. The SRB-VGG19 Dense ANL model was designed to address the adversarial data samples generated by FGSM attack.

Mathematical modelling of SRB-VGG19

The proposed SRB-VGG19 model initiates by collecting 4250 flower images with 850 flowers each from five classes of flowers such as daisy, dandelion, sunflower, tulip and rose from publicly available Flowers recognition dataset for classifying the fiver flower species and the images are denoted in the Eq. (1).

$$FLO_{4250} = \mathop \sum \limits_{F = 1}^{5} \bigcup\limits_{W = 1}^{850} {\left\{ {\mathop \sum \limits_{e = 1}^{255} \mathop \sum \limits_{d = 1}^{255} FLO_{edf} } \right\}}$$
(1)

where \({{FLO}_{00}}_{1}\) denotes single flower image \({{CTLS}_{00}}_{1}\) as in the Eq. (2).

$${{FLO}_{00}}_{1} =\left[\begin{array}{c}FLO(0,0)\\ \begin{array}{c}\begin{array}{c}FLO(\text{1,0})\\ \vdots \end{array}\\ FLO(\text{255,0})\end{array}\end{array} \begin{array}{c}FLO(\text{0,1})\\ \begin{array}{c}\begin{array}{c}FLO(\text{1,1})\\ \vdots \end{array}\\ FLO(\text{255,1})\end{array}\end{array} \begin{array}{c}\cdots \\ \begin{array}{c}\begin{array}{c}\dots \\ \dots \end{array}\\ \dots \end{array}\end{array} \begin{array}{c}FLO(\text{0,255})\\ \begin{array}{c}\begin{array}{c}FLO(\text{1,255})\\ \vdots \end{array}\\ FLO(\text{255,255})\end{array}\end{array}\right]$$
(2)

The flower images are applied to data processing module to generate the RBMCV flower images.

Problem formulation of unsupervised network feature extraction

The input data is processed to form the sobel filtered image to form the sobel filtered vector of flower images with the following steps.

Step 1 The flower image \({"FLO}^{IN"}\) input is denoting as vector with the pixel property mentioned in the Eq. (3).

$${FLO}^{IN}=\left\{POS\left(pos\right), Pixel(pix)\right\}$$
(3)

Step 2 Get the row pixel \(``RowPixel"\) and column pixel \(``columnPixel"\) values of each image in the Flowers recognition dataset as in (4) and (5). Here \({FLO}_{{00}_{1}}^{IN}\) denotes single flower image.

$$RowPixel={CFLO}_{{00}_{1}}^{IN}. RowPixel\left(pix\right)$$
(4)
$$ColumnPixel={CFLO}_{{00}_{1}}^{IN}. ColumnPixel\left(pix\right)$$
(5)

Step 3 Find the horizontal gradient \(``{G}_{h}"\) of the flower image. The horizontal gradient was calculated by taking differences in the flower image values between the columns as in (6).Here \(AColumn\) and \(BColumn\) denotes the first and second column of the \(ColumnPixel\) flower image respectively. The horizontal gradient \(``{G}_{h}"\) should be normalized before processing as in (7).

$$BColumn\left(j,k\right)=AColumn\left(j,k+1\right)-A(j,k-1)$$
(6)
$$\frac{BColumn\left(j,k\right)-{BColumn}_{min}}{{BColumn}_{max}-{BColumn}_{min}}\times 255=BColumn\left(j,k\right)$$
(7)

Step 4 Find the vertical gradient \(``{G}_{v}"\) of the flower image. The vertical gradient was calculated by taking differences in the flower image values between the rows as in (8).Here \(ARow\) and \(BRow\) denotes the first and second column of the \(RowPixel\) flower image respectively. The vertical gradient \(``{G}_{v}"\) should be normalized before processing as in (9).

$$BRow\left(i,k\right)=ARow\left(i,k+1\right)-A(i,k-1)$$
(8)
$$\frac{BRow\left(i,k\right)-{BRow}_{min}}{{BRow}_{max}-{BRow}_{min}}\times 255=BRow\left(i,k\right)$$
(9)

Step 5 Compute the final gradient \(``Grad"\) as in the Eq. (10).

$$Grad= BRow\left(i,k\right)+BColumn\left(j,k\right)$$
(10)

Step 6 Predefine the Threshold value of the pixel and the mark the edge features accordingly as in the Eqs. (11) and (12).

$$Threshold=1$$
(11)
$$If \,Grad>Threshold,\, then\, Mark\,\left(Pixel\left(pix\right)\right)=Edge$$
(12)

Step 7 Connect all the \(Edge\) to form the Sobel Filtered Image \(``SobelImage"\) as in (13)

$$SobelImage=Join(Edge)$$
(13)

Problem formulation of RBM component vectorized image

The input sobel filtered image is processed to form the RBMCV flower images with the following steps. RBM is an unsupervised two layered ANN that extracts the essential component from the image using probabilistic stochastic maximum likelihood function.

Step 1 Let the input \({SobelImage}^{IN}\) denoted as vector with the pixel property as in (14).

$${SobelImage}^{IN}=\left\{POS\left(pos\right), Pixel(pix)\right\}$$
(14)

Step 2 Identify the visible and invisible pixels of the each image from \({SobelImage}^{IN}\) . Let \({SIm}_{{00}_{1}}^{IN}\) represents the sobel filtered flower image. As the images as two dimensional, each image will have an alpha channel and each pixel in the image will have an alpha values either ‘0’ or ‘1’. If the alpha value of the pixel is ‘0’, then the pixel is invisible pixel. If the alpha value of the pixel is ‘1’, then the pixel is visible pixel and is denoted from the Eqs. (15) to (19).

$$Pix\left({ SIm}_{{00}_{1}}^{IN}\right)=P$$
(15)
$${\propto }_{a}\left(P\right)=\sum_{i=0}^{255}{\propto }_{b}\left(P\right)- {\propto }_{c}(P) [1-{\propto }_{b}\left(P\right)]$$
(16)
$${C}_{a}\left(P\right)=\frac{\left[{C}_{b}{\propto }_{b}\left(P\right)\pm {C}_{c}{\propto }_{c}\left(P\right)\left[1-{\propto }_{b}\left(P\right)\right]\right]}{{\propto }_{a}\left(P\right)}$$
(17)
$$VPixel=\sum_{l=0}^{255}{\propto }_{a}\left(P\right)$$
(18)
$$InVPixel=1-\sum_{l=0}^{255}{\propto }_{a}\left(P\right)$$
(19)

Step 3 Make the visible pixels \(``VPixel"\) as input nodes \(``In"\) and invisible pixels \(``InVPixel"\) as hidden nodes \(``Hn"\) of the RBM network as in (20) and (21).

$$In=VPixel$$
(20)
$$Hn=InVPixel$$
(21)

Step 4 Find the energy \(``Energy\left(In,Hn\right)"\) of the RBM network as in (22).

$$Energy\left(In,Hn\right)= \sum_{i}{a}_{i}{In}_{i }- \sum_{j}{b}_{j}{Hn}_{j }-\sum_{i,j}{a}_{i}{In}_{i }{Hn}_{j }{W}_{ij}$$
(22)

The \(``{a}_{i}"\) denotes activation function of node, denotes \(``{b}_{i}"\) bias function of node and \({"W}_{ij}"\) denotes the weight of the node.

Step 5 Find the RBM component vector \(``RBMCV"\) of the image using the Boltzmann distribution, Gibbs sampling and Contrastive Divergence step as in (23) to (27).

$$P\left({In}_{i }=1 \right| Hn)=\frac{1}{1+ {e}^{(-({a}_{i}+{W}_{ij }{Hn}_{j }))}}= {a}_{i}+ \sum_{i,j}{Hn}_{j }{W}_{ij}$$
(23)
$$P\left({Hn}_{i }=1 \right| In)=\frac{1}{1+ {e}^{(-({b}_{j}+{W}_{ij }{In}_{i }))}}= {b}_{j}+ \sum_{i,j}{In}_{i }{W}_{ij}$$
(24)
$${\Delta }W_{ij } = In_{0 } \otimes P{(}Hn_{0 } {|} In_{0 } ) - In_{k } \otimes P{(}Hn_{k } {|} In_{k } ) - In_{k }$$
(25)
$$New{W}_{ij }= Old{W}_{ij }+ {\Delta W}_{ij}$$
(26)
$$RBMCV=\sum_{i,j}New{W}_{ij}$$
(27)

Proposed SRB-VGG19 modelling

The RBMCV flower image \(RBMCV\) input data was processed with existing CNN models to choose the best CNN model. According to experiment results, VGG19 and DenseNet can classify floral species with an accuracy of above 80%. So, the VGG19 and DenseNet were selected to fine tune its structure to design the proposed SRB-VGG19 FCL and SRB-VGG19 Dense model. The five convolutional blocks are same both in SRB-VGG19 FCL and SRB-VGG19 Dense model. The SRB-VGG19 model initiates first two convolutional blocks two kernel filters and single max pooling followed by three convolutional blocks with four kernel filters and single max pooling. The convolution block operates with the following steps.

Step 1 Retrieve the \(``RBMCV"\) input image. Let denotes \(``W"\) Width, \(``H"\) denotes Height and denotes \(``D"\) Depth of the image with the number of filters as \(``K"\), spatial extent as \(``F"\), the stride as \(``S"\) and the padding as \(``P"\). Then it forms the new image with volume as in (28) to (30).

$$Wnew=\frac{W-F+2P}{S+1}$$
(28)
$$Hnew=\frac{H-F+2P}{S+1}$$
(29)
$$Dnew=K$$
(30)

Step 2 With the \(``RBMCV"\) input image and kernel as \(``K"\). The value \(``m"\) represents rows and \(``n"\) represents columns. The convolutional layer forms the numerical representation of the image that allows the neural network to extract the essential patterns from the image. The convolution of 3X3 kernel filter forms the output as in (31).

$${RBMCVCon}_{ij }={(RBMCV\times K)}_{ij }= {\sum }_{x=0}^{m-1}{\sum }_{y=0}^{n-1}{RBMCV}_{i+x,j+y}{K}_{x,y}$$
(31)

Step 3 The \(``RBMCVcon"\) convolutional image was then passed through 2X2 max pooling. The pooling operation was done to reduce the spatial image size to reduce the computation time. Let denotes \(``Wc"\) Width, \(``Hc"\) denotes Height and denotes \(``Dc"\) Depth of the \(``RBMCVcon"\) convolutional image with the spatial extent as \(``Fc"\) and the stride as \(``Sc"\) as in (32) to (37). The Dimension of the pooled RBMCV image is shown in (38).

$$Wc=Width(RBMCVcon)$$
(32)
$$Hc=Height(RBMCVcon)$$
(33)
$$Dc=Depth(RBMCVcon)$$
(34)
$$Wcnew=\frac{Wc-Fc}{Sc+1}$$
(35)
$$Hcnew=\frac{Hc-Fc}{Sc+1}$$
(36)
$$Dcnew=Dc$$
(37)
$$Dim\left(RBMCVPol\right)=Wcnew\times Hcnew\times Dcnew$$
(38)

The step 2 and step 3 was repeated for five convolutional block, the final pooled image \(``RBMCVPol"\) was formed.

Proposed SRB-VGG19 FCL problem formulation

In SRB-VGG19 FCL model, the pooled image \(``RBMCVPol"\) was given to two FCL layers that were used to make the prediction from the extracted features. The FCL layer is a MLP that maps the activation volume of the \(``RBMCVPol"\) image from the combination of previous layers into a class probability function. The volume of \(``RBMCVPol"\) image represents \(``Wp"\) Width, \(``Hp"\) denotes Height and denotes \(``Dp"\) Depth of the image. For FCL layer, the depth of the connection will be \(``l-1"\) having \(``l"\) layers in the FCL layer network. The output of the FCL layer performs the mapping \({m}_{a}^{(l-1)} \times {m}_{b}^{(l-1)}\times {m}_{c}^{\left(l-1\right)}\) of the activation volume from the previous layers in the form of probability distribution. Here \(``i"\) denotes number of layers in FCL network and output of the FCL layer is shown in (36) and (37).

$${y}_{i}^{(l)}=F( {RBMCVPol}_{i}^{(l)})$$
(36)
$${RBMCVPol}_{i}^{(l)}=\sum_{x=1}^{{m}_{a}^{(l-1)}}\sum_{y=1}^{{m}_{b}^{(l-1)}}\sum_{z=1}^{{m}_{c}^{\left(l-1\right)}}{Wp}_{i,x,y,z}^{\left(l-1\right)}{({y}_{i}^{\left(l-1\right)})}_{y,z}$$
(37)

Proposed SRB-VGG19 dense model problem formulation

In SRB-VGG19 Dense model, the pooled image \(``RBMCVPol"\) was given to two Dense block crafted with noise using ANL in the intermediate layer that were used to make the prediction from the adversarial FGSM data sample features. Each dense block contains four convolutions, where each single convolution layer contains a batch normalization, ReLU and single convolution followed by concatenation of the feature map. So, the four convolutional layer forms concatenated together to form the four layered Dense block. Assume the input given to the dense block is \(``RBMCVPol"\). The four layered dense block is formed as in (38) to (42).

$$Let x= RBMCVPol$$
(38)
$${Y}_{1}={F}_{1}( x)$$
(39)
$${Y}_{2}={F}_{2}( x, {F}_{1}( x))$$
(40)
$${Y}_{3}={F}_{3}( x, {F}_{1}\left( x\right),{F}_{2}( x, {F}_{1}( x)) )$$
(41)
$${Y}_{4}={F}_{4}( x, {F}_{1}\left( x\right),{F}_{2}\left( x, {F}_{1}\left( x\right)\right),{F}_{3}( x, {F}_{1}\left( x\right),{F}_{2}( x, {F}_{1}( x)) ) )$$
(42)

Here \({"Y}_{1},{Y}_{2},{Y}_{3},{Y}_{4}"\) represents the output of first convolution, second convolution, third convolution, fourth convolution respectively.

Problem formulation of FGSM attack ANL module of SRB-VGG19 dense ANL

In SRB-VGG19 Dense ANL model, the pooled image \(``RBMCVPol"\) was given to two Dense block that were used to make the prediction from the extracted features. Each dense block contains four convolutions, where each single convolution layer contains a batch normalization, an intermediate ANL layer that process in two cycles followed by ReLU and single convolution followed by concatenation of the feature map. So, the four convolutional layer with ANL layer are concatenated together to form the four layered Dense ANL block. The FGSM attack was addresses by SRB-VGG19 Dense ANL model. The adversarial noise are samples are created using FGSM attack. Here \({{"FLO}_{00}}_{1}"\) is the original single flower image. The \({{"AdvFLO}_{00}}_{1}"\) is the single adversarial noise flower image. Let us see how the adversarial noise data sample is created by FGSM attack for a single image in (43).

$${{AdvFLO}_{00}}_{1}={{FLO}_{00}}_{1}+ \in \times sign( {\nabla }_{{{FLO}_{00}}_{1}}Z(\varphi , {{FLO}_{00}}_{1},Label)$$
(43)

Here \(``\in "\) denotes hyper parameter that controls the magnitude level of the noise, \(``\varphi "\) denotes SRB-VGG19 Dense ANL parameters, \(``Label"\) denotes the true flower class label of the input dataset and \(``Z"\) denotes the cost function of the SRB-VGG19 Dense ANL model. The ANL layer was added inside the dense block. Assuming \({" X}_{0}"\) as the input of the first convolutional layer of the dense block which is \(``RBMCVPol"\) as in (44). The number of layer is \(``t"\) which is four layer and \({"x}_{t}"\) is the output of the \({"t}^{th}"\) layer. \({"Y}_{t}^{-}"\) denotes the sub-network from \({"t+1}^{th}"\) layer to the last layer of \({"Y}_{t}^{+}"\) in ANL layer which was based on the equation from (45) to (47).

$${X}_{0}=RBMCVPol$$
(44)
$${x}_{t}=Output of{ Y}_{t}^{-}$$
(45)
$${x}_{t}=Input of{ Y}_{t}^{+}$$
(46)
$${\eta }_{t}=Adversarial Noise$$
(47)

The Adversarial noise \({"\eta }_{t}"\) was created by FGSM method that was shown in (48). Here \(``r"\) denotes the random scalar that controls the magnitude level of the noise, \(``{s(x}_{t})"\) dentotes the standard deviation of the input \({"x}_{t}"\), method, \(``\oint "\) denotes the Gaussian distribution of hyper parameter and \({"\nabla }_{{x}_{t}}"\) denotes the gradient of the \({"t}^{th}"\) layer. The ANL Layer simply adds the Adversarial noise \({"\eta }_{t}"\) to the input \(``{x}_{t}"\) to generate \(\widehat{{"x}_{t}}"\) that is passed to the next cycle2 of the ANL layer.

$${\eta }_{t}=r\times {s(x}_{t})\times {\nabla }_{{x}_{t}}\widehat{\text{Z}}$$
(48)
$$r={Clip}_{<0,\in >}\left\{\oint (\frac{\in }{2}, {(\frac{\in }{4})}^{2}\right\}$$
(49)
$${\nabla }_{{x}_{t}}={\nabla }_{{x}_{t}}Z(\varphi , {{FLO}_{00}}_{1},Label)$$
(50)
$$\widehat{{x}_{t}}={x}_{t}+{\eta }_{t}$$
(51)

SRB-VGG19 implementation results and discussions

To categorize the five flower species, the proposed SRB-VGG19 starts by gathering 4250 flower images (https://www.kaggle.com/datasets/alxmamaev/flowers-recognition) from the openly available Flowers Recognition KAGGLE dataset. The daisy, tulip, rose, sunflower, and dandelion are the five groups of flower species that have been used in this study. The Flower Recognition dataset containing 4250 flower samples with 850 samples of daisy, sunflower, rose, tulip and dandelion was used for implementation with 3400 training sample and 850 testing samples. The sample images from the dataset are shown in the Fig. 12. The tensorflow compatible images are formed from original images and then grayscaled image are formed and the results are shown in Fig. 13.

Fig. 12
figure 12

Sample flower class images from flowers recognition dataset.

Full size image
Fig. 13
figure 13

Tensorflow compatible and grayscaled image results from dataset.

Full size image

From the grayscaled image, the sobel filtered image was formed by finding the prewitt vertical and horizontal kernel and the results are shown in Fig. 14.

Fig. 14
figure 14

Prewitt Vertical and Horizontal kernel image results from Grayscaled image.

Full size image

From the prewitt vertical and horizontal kernel images, the sobel filtered images are formed. From the sobel filtered images, the RBM component vectorized flower images are formed and the results are shown in Fig. 15.

Fig. 15
figure 15

Sobel filtered and RBMCV flower image results from Prewitt vertical and horizontal kernel image.

Full size image

The Feature maps results obtained from the two convolutional blocks and two dense blocks of SRB-VGG19 Dense model is shown in Figs. 16, 17, 18, 19, 20, 21, and 22.

Fig. 16
figure 16

Feature map results from first convolutional block of SRB-VGG19 dense mode1.

Full size image
Fig. 17
figure 17

Feature map results from second convolutional block of SRB-VGG19 dense mode1.

Full size image
Fig. 18
figure 18

Feature map results from third convolutional block of SRB-VGG19 dense mode1.

Full size image
Fig. 19
figure 19

Feature map results from fourth convolutional block of SRB-VGG19 dense mode1.

Full size image
Fig. 20
figure 20

Feature map results from fifth convolutional block of SRB-VGG19 dense mode1.

Full size image
Fig. 21
figure 21

Feature map results from first dense block of SRB-VGG19 dense mode1.

Full size image
Fig. 22
figure 22

Feature map results from second dense block of SRB-VGG19 dense mode1.

Full size image

The RBMCV flower images are fitted to existing CNN models to choose the best CNN. The performance was analysed that is shown in Table 2 and Fig. 23. The RBMCV flower images are fitted to SRB-VGG19 FCL model and SRB-VGG19 Dense model prediction results are shown in Fig. 24.

Fig. 23
figure 23

Performance analysis of SRB-VGG19 dense mode1.

Full size image
Fig. 24
figure 24

Prediction results of SRB-VGG19 dense mode1.

Full size image

Security countermeasure FGSM attack implementation results and discussion

The SRB-VGG19 Dense ANL model was validated by creating the adversarial noise samples using FGSM method. The implementation results of the FGSM noise data samples created in the SRB-VGG19 Dense ANL model is shown in Fig. 25 (Table 3).

Fig. 25
figure 25

Adversarial noise sample results with FGSM attack in SRB-VGG19 dense mode1.

Full size image
Table 3 Performance Analysis of SRB-VGG19.
Full size table

The performance analysis of SRB-VGG19 Dense ANL model was done by giving the adversarial noise sample input with varying gradients of \(\nabla\) = 0, \(\nabla\) = 2 and \(\nabla\) = 4. The perturbation magnitude of FGSM noise sample magnitude was varied with \(\in =0.00, 0.01, 0.10, 0.15, 0.20, 0.25\) and defended accuracy of SRB-VGG19 Dense ANL model was analyzed using 10 k-Fold cross validation and mean performance metrics are shown in Table 4. The error analysis of misclassified instances of the proposed SRB-VGG19 Dense ANL model was done by considering with and without the ANL layer and the analysis is shown in Table 3.

Table 4 Performance Analysis of SRB-VGG19 Dense ANL Model.
Full size table

The performance of the proposed SRB-VGG19 Dense ANL was compared with the existing works that performs flower species classification and it is shown in Table 5.

Table 5 Performance Analysis of SRB-VGG19.
Full size table

Computational complexity of proposed SRB-VGG19 dense ANL model

The computational complexity \(``CCom"\) of SRB-VGG19 Dense ANL model depends on the number of input and output dimensions and is shown in (52) where \({"N}_{In}"\) is the number of inputs of the proposed model and \({"N}_{Out}"\) is the number of outputs of the proposed model. The number of the inputs depends on the height \(``H"\) and width \(``W"\) of the feature map and number of input channels \(``{C}_{In}"\) as in (53). The number of the outputs depends on the kernel size \(``K"\) of the feature map \({"F}_{map}"\) and number of output channels \(``{C}_{Out}"\) as in (54).

$$CCom=\Theta {(N}_{In}+{N}_{Out})$$
(52)
$$\Theta {(N}_{In})=H \cdot W \cdot {C}_{In}$$
(53)
$$\Theta {(N}_{Out})=K \cdot K \cdot {C}_{Out}$$
(54)

Now the proposed model is refined by adding the ANL in the dense layered block and its complexity is shown in (55) with \(``\in "\) is the perturbation magnitude that controls the scale of noise and \(``\delta "\) is the adversarial noise generated by the proposed model.

$${N}_{ANL}={F}_{map}+ \in \cdot \delta$$
(55)

The dataset bias of the proposed model should be considered for enhancing the robustness and the model generalization. The presence of imbalanced dataset and sampling bias may lead to model overfitting issues. However, the presence of ANL layer in the dense layered block mitigates the sampling bias and imbalanced datasets that ensures the robustness during training. The impact of adversarial defenses on model generalization is performed by analyzing the performance of the proposed model with various is the perturbation magnitude " ∈ " that controls the scale of noise and it is shown in Table 3. Since the proposed SRB-VGG19 Dense ANL model introduces noise, it could prevent the model from becoming highly sensitive to small data fluctuations, thus improving the model’s ability to generalize to new unseen inputs. Also, as the ANL layer involves techniques that mask gradients during backpropagation, it could make the model harder for attackers to manipulate the input data during training, as the model might learn less directly from the data thereby reducing the generalization. The input flower image was fine-tuned to RBM Component Vectorized (RBMCV) flower images that was received as the input for SRB-VGG19 Dense ANL model. Four layered dense block was added as the last layer of the existing VGG19 model to design proposed SRB-VGG19 Dense ANL model.During fine-tuning, the weights of dense layers are adjusted to handle the embeddings that have been modified by the Adversarial Noise Layer. The loss function was fine-tuned to cross-entropy loss for flower species classification, where the goal is to minimize the difference between the predicted class probabilities and the true labels that ensures the robustness to adversarial perturbations. The fine-tuning of the proposed model was finally done by testing the model with various noise scale to assess the performance.

Conclusion and future enhancements

This research attempt to categorize the flower species by analysing the essential features of the flower image and its edges. This study’s primary goal is to provide the effective data pre-processing through unsupervised learning by applying the Sobel filter and RBM neural network component to the input data image. The major contributions of this research are summarized into four ways. The initial contribution focuses on preparation of the dataset to enhance the extraction of flower essential features using unsupervised learning by influencing the Sobel filter technique to form the Sobel Filtered vectorized flower image. As the final outcome information is unknown, a spatial intensity image must be prepared in order to extract only the most significant information from the input image. So the Sobel filtered flower image was processed with RBM Neural network that learns the probability distribution of the image pixel thereby extracting the most essential features to form the RBMCV flower images. The second contribution involves optimizing the Dense block by creating the four layered dense block for adding into SRB-VGG19 Dense model. The third contribution is the fine-tuning of VGG19 and DenseNet to design the proposed SRB-VGG19 with the sub model design. The final contribution of this research deals with the security countermeasure towards data poisoning FGSM attack by designing SRB-VGG19 Dense ANL model.

As an overview of novelty, the first proposed SRB-VGG19 FCL model retrieves the RBMCV flower feature maps and processed with two FCL layers along with an third additional FCL layer to generate optimized RBMCV feature map. The third updated Dense Layer block consists of one Batch Normalization, one ReLU, and one dropout layer. The second SRB-VGG19 Dense model retrieves the RBMCV flower feature maps and processed with two Dense block to generate optimized RBMCV feature map. The addition of four layered Dense block overcomes the problem of vanishing gradient. The dense blocks are composed of four convolution blocks connected in series. The single convolutional block in the dense block consists of batch normalization, RELU and convolution that allow cross-layer connections between two non-adjacent convolutions. Each input convolutional block contains feature maps from all the previous convolutional blocks. Finally, both the SRB-VGG19 FCL and Dense model ends with the softmax layer with five neurons to classify the five flower species. The third SRB-VGG19 Dense ANL Model overcome the data poisoning FGSM attacks that are vulnerable to Adversarial data poisoning attack. The ANL layer was created as the hidden layer in between the Convolutional layers of the four layered dense block. The ANL layer was designed inside the hidden layer of the dense block. The proposed SRB-VGG19 model faced challenges with data pre-processing while forming the RBMCV flower image and choosing the optimal CNN to improvise the model accuracy. It also faced challenge during the creating of the ANL layer as two cycles to predict the ideal flower samples even the model receives adversarial FGSM noise data samples. The Flower Recognition dataset containing 4250 flower samples with 850 samples of daisy, sunflower, rose, tulip and dandelion was used for implementation with 3400 training sample and 850 testing samples. The training data was converted to grayscaled flower image, then to sobel filtered followed by RBMCV flower images.

The RBMCV flower images were fitted with existing CNN models to analyse the performance for selecting optimum CNN. According to experiment results, VGG19 and DenseNet can classify floral species with an accuracy of above 80%. So, VGG19 and DenseNet were integrated and fine-tuned to design the proposed SRB-VGG19 model. When compared to the current deep learning models, the implementation results show that the proposed SRB-VGG19 Dense Model classifies the flower species with a high accuracy of 98.65%. Now to test the security countermeasure of data poisoning FGSM attack, the adversarial FGSM data samples was created and attached with original image for performance evaluation. As the SRB-VGG19 Dense ANL model was designed with ANL as the intermediate layer in the dense block, the performance degradation was least negligible which is acceptable. When the FGSM samples are varied with magnitude and gradient, the accuracy of flower species classification was varied from 98.65% to 98.12, 98.43, 98.24, 98.33, 98.02 and 98.45. High-dimensional feature maps created by VGG19’s convolutional layers serve as embeddings of the input image. The input image is converted into a high-dimensional feature space by the convolutional layers, which act as the embedding layer. Feature maps that capture different aspects of the image are produced by each convolutional process. The original image’s embeddings are the feature maps that the convolutional layers create. For the purpose of analyzing and categorizing the flower species, these embeddings offer a condensed representation of the image. The convolutional layers learn to highlight more significant features that are less vulnerable to adversarial manipulation as the model trains with adversarial noise, making the embeddings they learn more resilient to the perturbations in the input data. The Adversarial Noise Layer plays a key role in modifying the learned embeddings. The model is compelled to learn more resilient embeddings that capture invariant features in spite of the noise by introducing adversarial noise into the input. By learning representations that are resistant to small, undetectable changes in the input image brought on by adversarial attacks, the embeddings learnt from adversarially augmented data aid in the model’s ability to generalize. The proposed approach uses the embeddings to generate final predictions, and accurate classification depends on the robustness of the embeddings. The model can accurately classify the flower species even when there is adversarial noise since the embeddings have been modified to be more resistant to adversarial attacks. The suggested SRB-VGG19 Dense ANL model performs exceptionally, but adjusting the ANL layers for high gradients, error functions, and optimizers is still a challenge. This challenge may be addressed in a future version of the study. As an extension of this study, the system may be validated for performance and the various types of data poisoning attacks may also be implemented for various noise data samples.