Clinically validated classification of chronic wounds method with memristor-based cellular neural network

Abstract

Chronic wounds are a syndrome that affects around 4% of the world population due to several pathologies. The COV-19 pandemic has enforced the need of developing new techniques and technologies that can help clinicians to monitor the affected patients easily and reliably. In this prospective observational study a new device, the Wound Viewer, that works through a memristor-based Discrete-Time Cellular Neural Network (DT-CNN) has been developed and tested through a clinical trial of 150 patients. The WV has been developed to serve as the state-of-art tool, capable to return the actual clinical information that is most needed by the caregivers: through the WBP scale, it classifies four classes of wounds by the type of tissue: A-only granular tissue; B-<50% slough; C->50% slough; D-necrosis. This work aims to describe in depth the technology and the computational techniques that have been implemented, and to demonstrate reliability in automatically identifying, classifying through internationally accepted clinical scales and measuring such wounds, that peaked to over a 90% of accuracy.

Introduction

Skin ulcers are a chronic pathological condition affecting around 4% of the world population. In Europe alone more than 10 million patients are affected by this syndrome with a yearly cost for Healthcare Systems of over 4 billion euros for their treatment¹. The highest incidence (over 60%) is observed in patients over 65 years of age and is commonly associated with pre-existing chronic diseases such as diabetes, vascular problems, heart disease and obesity which cause problems to the circulatory system in various districts of the body². This syndrome, if not treated properly, may result in necrosis of the whole limb due to infections and must be then treated surgically^3,4,5. Studies have showed that protocols that enable easy and continuous monitoring through standardized data are a key factor in avoiding such complications^6,7.

This problem related to correct wound monitoring has been studied by different research groups, which, by developing artificial intelligence (AI) algorithms have showed the reliability of such tools in the field of wound care⁸. Recent studies, such as the ones conducted by Cross et al.⁹ and Anisuzzaman et al.¹⁰ have demonstrated the capability of automatically identifying and determining the aetiology of the wounds through advanced computational means. Though the increment of the similar projects initiated in this field in recent years, few have been finalized in developing a usable tool for caregivers, that can calculate effective and standardized clinical data regarding the actual state of the wound, and even fewer have reached effective clinical usage.

Our approach, developing the Wound Viewer (WV), is based on the direct needs of the care givers in terms of providing them with a device capable of gathering and computing standardized clinical data that is commonly used in wound care such as the morphological measurement of the wound and its clinical classification through Wound Bed Preparation (WBP) Score, TIME and TEXAS scales^11,12,13. To do so, WV implements a neuromorphic network which exploits memristor theory¹⁴. Memristors are non-linear circuit elements that can change their internal electric resistance in a non-volatile fashion through charge and flux (i.e. the integrals of the current flowing through the element and the integral of the voltage applied to the same). Thanks to their dynamics memristors are usually implemented as synapses in brain-like computational networks. In our specific case a memristive Discrete-Time Cellular Neural Network (DT-CNN) has been developed, taking inspiration from the work of Itoh and Chua¹⁵. Our DT-CNN can be assimilated to memristive Cellular Automaton (CA)¹⁶ which is known to be a powerful tool for image processing, accordingly modified in order to implement a Belief Propagation Inspired (BPI) algorithm for wound recognition and analysis from wound images taken by the device in real time^17,18,19. Exploiting the afore mentioned memristive dynamics, and the mathematical implementation of the BPI through our DT-CNN it was possible to generate a methodology that could be easily trained to detect a wound from a generic image and classify that wound through validated clinical scales.

In this work we will describe in details the computational methodology, proving its reliability through the results of a clinical trial conducted with the WV. Moreover, we will provide information regarding its common usage, since its entrance in everyday clinical usage, in order to prove the efficacy of neuromorphic systems on the field for wound care specialists.

Materials and methods

The WV device has been developed due to the need to acquire wound images and classify them in an automated and precise way. It is a customised AI medical device, that runs a proprietary AI algorithm for wound measurement and assessment. The reliability of the device has been demonstrated through the results of the clinical trial with the protocol number OC 15194, identified by ethics committee approval of the Ethical Committee of the Azienda Opedaliera Universitaria San Luigi Gonzaga (Orbassano, Italy). During the study, a population of 150 patients has been considered statistically significant considering that generally for clinical trials of this type, the recruited patients are around 30, for a good statistical basis^20,21. With this hypothesis, to statistically represent each aethiology, we considered 3 cohorts with 50 subjects each. The device, shown in Fig. 1a, is made up of a custom-designed electronic board mounting a five-megapixel color CMOS camera sensor for high resolution pictures, 16 high precision infrared (IR) distance sensors and four white LEDs with the scope of eliminating shadows from the photographs taken by the device.²².

Fig. 1

(a) Front and rear view of the WV device with its components. (b) Wound detection algorithm: the workflow shows the steps that the device follows to classify the wound. Starting taking the photo of the lesion, the first part of the algorithm elaboration identifies the Region of Interest (ROI) that contours the wound in the image. The second part consists of the segmentation and subsequent classification of the wound inside the ROI. The first part of algorithm consists in a CNN of a two-dimensional layer with 24 convolutional layers. The second part is described as a DT-CNN based on the memristive cells of the CA and the BPI algorithm.

The LEDs are placed at a distance of 5 cm from the CMOS camera. Moreover, these light sources are set to a standard intensity, ensuring uniformly lit pictures which can be analysed against a standard color scheme. Operators control the device through a special front end via a capacitive touch screen display²². The following four steps describe the normal use protocol of WV when analysing a wound:

• Point the device towards the wound, keeping it parallel to the surface;

• The 16 IR distance sensors calibrate the focal ratio of the camera;

• Automatic identification of the wounds in the picture through ROIs surrounding the lesions;

• The wounds in the ROIs (Regions of Interest) are analysed by the algorithm;

• All relevant information computed is shown on the display (wound area, depth, tissue segmentation and WBP score classification). WBP is a scale that classifies wounds in four classes depending on the type of tissue that composes its bed: A-only granular tissue; B-<50% slough; C->50% slough; D-necrosis present. Refer to¹¹ for further clarification on WBP score wound classification.

The WV detects the wound and counts the pixels of the resulting analysis. The number of pixels is then multiplied by the focus distance of the CMOS camera (i.e. the area in [$cm^2$] covered by a single pixel with respect to the distance from which the picture has been taken, read through the IR sensors shown in Fig. 1a).

The parameters computed by the device (i.e. the morphological features and its classification) play a key role the determination of wound diagnosis and the expected healing time. The device and its software have been integrated with a secure cloud system which allows the operator to share the pictures with the rest of the medical team so to provide an eventual remote clinical consultation²². Finally, the device implements a compliant Electronic Medical Record (EMR) allowing the wound to be monitored as it evolves allowing the physicians to immediately verify how effective the therapy is through a quantitative series of indicators²³. The device also implements 16 IR sensors measure distances concurrently as the CMOS camera takes the digital photographs. These distance measurements are fundamental in calculating indicators of the wound as the depth and area of the patient’s wound. The distances taken by the IR sensors are converted into a coefficient through a calibration mapping uploaded onto the device during manufacture²².

Regarding its neuromorphic methodology for wound analysis, the algorithms are composed by two sub-networks coupled in a waterfall configuration. Both networks have been implemented by means of a software simulated algorithm; indeed, they are based on two different architectures. The first, as shown in Fig. 1b has the scope of detecting and extracting the ROIs from the image through a multi-layered convolutional neural network²⁴. This is composed by a two-dimensional layer of locally connected cells with same dimension of the images that are processed followed by two blocks of convolutional layers, responsible for image feature extraction for automatic wound detection. The first block is composed of 24 convolutional layers each of 512 cells, followed by a ReLU activation function. The second block is instead composed of 24 convolutional layers each of 128 neurons, also followed by ReLU. In the middle of the two blocks is posed one more fully connected layer for data transfer amongst them. Part of the architecture is composed by one last fully connected layer of cells that functions as a readout/softmax layer for classification.

In addition, the used CNN is described by the following features:

• 1. A stride of 2 was used, which reduced the spatial dimensions of the feature maps, providing a balance between computational efficiency and feature extraction.

• 2. Kernel: it utilizes a 9x9 kernel size, which allows the network to capture more complex patterns by covering a larger area of the wounds.

• 3. Padding involved adding the “same” padding, which ensured that the output feature map had the same spatial dimensions as the input, preserving edge information.

• 4. Regarding the choice of the hyperparameters, the most commonly used optimization algorithms is the random search. After several trials, where different parameters (number of filters, filter size, pooling size, dropout rate, learning rate, batch size, epochs) have been combined to test different models, the final model has been chosen through the best obtained results performance.

The neural network was trained on a dataset containing around 1.500 images, coming from open source datasets²⁵ and from previously collected images. For training purposes, the wounds depicted in the images were manually segmented and classified according to the WBP scale, resulting in an even distribution across the four possible classes of wounds. The final training dataset Therefore, included various image types relevant to the classification task. It is necessary to point out that it was made sure that the dataset included images of patients with the broadest possible range of skin tones and characteristics in order to eliminate possible biases and misclassifications. The set was subsequently divided into training (70%), validation (15%), and test (15%) sets. To increase the diversity of the training data and improve the model’s generalization, several data augmentation techniques have been used: random rotation (up to 20 degrees), horizontal flipping, random cropping, and normalization. These techniques help the network become more robust to variations in the input data and increase the overall number of images. During training, the validation set was used to monitor the model’s performance after each epoch. Early stopping was employed to prevent overfitting, where training stops if the validation loss does not improve for 10 consecutive epochs. Accuracy and loss were the primary metrics used for validation, ensuring that the model was not just memorizing the training data but generalizing well to unseen data. Hyperparameter tuning was conducted using grid search. Parameters such as learning rate, batch size, and dropout rate were tuned to optimize performance. For instance, learning rates were tested between 0.001 and 0.0001, and dropout rates were varied between 0.3 and 0.5 to find the best configuration. The best-performing model was achieved with a learning rate of 0.0001, a batch size of 64, and a dropout rate of 0.4.

The second sub-network consists of a DT-CNN computing architecture applied on the ROIs previous extracted, to segment the wound and provide relevant measurements and analyses. The processing units making up DT-CNNs are called neurons or cells. Once the entire surface of the wound has been recognised, the highlighted elements are analysed by color scheme. All possible pixel color sets forming the wound were classified into four macro groups: red, white, black and yellow identifying the different possible tissues composing the wound bed and its feature. During the network’s training phase, the cellular nonlinear network in the WV algorithm, which processes a two-dimensional color image, is provided with statistical information about the tissue forming the wound bed through color analysis. The wound images in the trial group were labelled using WBP score and the algorithm then matched the color schemes read within the area of the image depicting the wound. The algorithm uses this test phase to analyse these color schemes and automatically classify them. In the following, a detailed insight of this second subnetwork (and of its training algorithm) is provided. After a brief introduction of the memristor theory underlying the working principle of the proposed work, and of the memristor-based CA representing the basic block of this CNN, we introduce the BPI algorithm, that served as inspiration for our training algorithm. Finally, the CA-based architecture with the BPI training algorithm will be described.

The source code for the wound analysis methodology described in the following sections, as implemented in the WV device, is consultable from Zenodo²⁶. Please refer to the Code Availability Statement at the end of the manuscript for further details on its accessibility.

Generic memristor synapses model

As mentioned in the previous sections, the networks developed for the analysis of the wounds, have been designed upon the implementation of memristor elements. Exploiting their synapse-like dynamics, their potential non-volatility and their ease of programming renders bio-inspired systems capable of performing complex computations. Such elements are well known in literature and the advantages deriving from their use have been widely studied, obtaining highly controllable mathematical models^27,28,29. The avant-garde of this device highlighted by recent works, especially in the functional electronic field, pushes towards the future goal of a possible concrete realization of fully memristor-based integrated hardware models enabling neuromorphic computing techniques^30,31. Although memristor arrays can perform parallel in-memory operations with significantly enhanced speed and energy efficiency, memristors have non-ideal intrinsic effects that can affect the operation of the device. Three main issues are highly noted: low yield and non-uniformity in memristor crossbar arrays, difficulty in matching the performance of software implementations due to device imperfections, and the time-consuming nature of convolutional operations, which leads to a speed mismatch between the memristor convolver and the memristor array used for vector-matrix multiplication. To overcome such imperfections, several strategies have been developed over the years to improving overall memristor-based system performance. Among them, Yao et al. required the integration of multiple memristor crossbar arrays with the aim of splitting the weights into different arrays for parallel computing³¹. Marrone et al. have used Dynamic Route Maps (DRMs) on memristive devices in order to characterize the single elements for implementation purposes³². In particular, in case PCMs would be integrated, the non-ideal memristors effects could affect the method due to other variables, such as elements’ temperature or leakage currents inside the cross bar³³. The presence of these elements that can have an effect on the internal state of each memristor. Therefore, a possible optimization strategy could be the implementation of additional circuits, as continuous element reset cycles, to consider the bias introduced by the non-linearity devices to obtain optimal results. These approaches have been implemented for CNNs, adapting the memristor arrays in the architectures: therefore, the same methods could be applied for the presented DT-CNN visioning future fully integrated architectures. It is therefore consequent that the choice of basing the presented system on memristive dynamics would serve as a solid starting point for future developments obtaining major efficiencies both computationally and energetically.

To provide a broader perspective, other works that utilizes memristors in analog neuromorphic neural network can be cited. For instance, Nikiruy et al. developed a neuromorphic circuit scheme with CMOS integrated HfO2-based memristive devices. The circuit has to handle association tasks, as blooming and pruning of the brain with memristive synapses. The authors, in this example, recreated the human learning scheme that learns from mistakes by eliminating from the initial surplus of synaptic connections those that lead to an undesirable outcome³⁴.

For the purpose of this work, it is necessary to part from the description of the Ideal Memristor Model (Fig. 2a), according to Corinto et al., is based on current and flux momentum (also named charge – q and flux – $\varphi$, respectively)^14,35,36.The memristance can be defined as $R(q) \equiv d \varphi (q) / d q$, so R is function of q. $\varphi (q)$ is the charge driven flux in the device and is considered the constitutive equation of the memristor. In addition, Ascoli et al. produced a mathematical model of a broader class of memristors, which has been already successfully implemented in other memristor-based neuromorphic algorithms^19,37:

$$\begin{aligned} \frac{d \textbf{x}}{d t}= & \textbf{f}(\textbf{x}(t), c(t), t), \end{aligned}$$

(1)

$$\begin{aligned} r(t)= & M(\textbf{x}(t), c(t), t) c(t), \end{aligned}$$

(2)

where $\textbf{x} \in \mathbb {R}^{n}$ denotes an internal state variable vector, $c(t) \in \mathbb {R}$ and $r(t) \in \mathbb {R}$ represent the current (voltage) input and current (voltage) output under charge (flux) control, respectively, $f(\cdot , \cdot , \cdot ): \mathbb {R}^{n} \times \mathbb {R} \times$ $\mathbb {R} \rightarrow \mathbb {R}^{n}$ is the state evolution function and $M(\cdot , \cdot , \cdot ): \mathbb {R}^{n} \times \mathbb {R} \times \mathbb {R} \rightarrow \mathbb {R}^{n}$ stands for the memristance (memductance) under charge (flux) control. In general, $g(\cdot , \cdot , \cdot )$ is not just a function of $q(\varphi )$ under charge (flux) control, because systems of this kind are characterized by a multivalued flux-charge relation¹⁹. This memristor model has been used to describe the neuron synapses of the DT-CNN algorithm units.

Memristive cellular automata

A DT-CNN, according to the work of Itoh and Chua, can be also considered as a memristor-based Cellular Automata (CA)¹⁵. From the work of Stephen Wolfram, a CA is a mathematical methodology that is based on a grid of simple computational elements called cells, connected in different fashions on a spatial grid (as depicted in Fig. 2b)^16,38. Each cell on the grid evolves at each time epoch depending on the actual states of the given cell and the surrounding cells and on an eventual external input. The cells of a CA can evolve into multiple discrete states or following a binary logic¹⁸. From Wolfram’s definition, a CA system is defined by the following elements:

• there must be a spatial representation of the involved entities;

• uniformity, or in other words all the entities must have the same characteristics and must be identical and interchangeable;

• locality, each entity changes its state from a generation to the other taking into account the states of the entities within a given surrounding radius.

Still, CAs can be described in the following fourfold: $<d, Q, N_{n}, f>$ where

• $\quad d$ is the dimension of the CA;

• $\quad Q$ is the space of the states which the cells can assume;

• $\quad N_{n}$ is the neighbourhood index which describes the region of influence of adjacent cell’s state change;

• $\quad f$ is the generation transition function which describes the state change of each cell at each time instant and must be a function of a cell neighbourhood described by parameter $N_{n}$.¹⁸

Fig. 2

(a) Memristor model schematic representation. I and V represent te current flowing through the device and the voltage applied. As described in the text these change over time due to the changes of its resistance (memristance), based on the momenta of both parameters (i.e. q(t) and $\varphi (t)$). (b) Simple representation of the connections of a cell in a CA. In the depicted case the state of the cell $y_{i,j}$ is dependant from its actual state, the states of a neighbourhood $N_n=9$ and external inputs (indexed as $u_{i+g,j+l}$) according to Eq. (3).

As mentioned, the equivalence of a memristor-based CA and a DT-CNN has been done by Itoh and Chua¹⁵ 2a through the following definition:

$$\begin{aligned} y_{i, j}(n T) =M\Biggl (\sum _{g, l \in \{-1,0,1\}} a_{g, l} y_{i+g, j+l}((n-1) T)+\sum _{g, l \in \{-1,0,1\}} b_{g, l} u_{i+g, j+l}((n-1) T)+\Delta q\Biggl ), \end{aligned}$$

(3)

where y is the output or the state of the cell and u are the external inputs of the system. T is the time period in which there is a new input and, therefore, a new generation change in the system, while $\Delta q$ represents the charge accumulation in the memristor during the previous generations. $a_{g, l}$ and $b_{g, l}$ are elements of two distinct matrices A and B which both have size $G \times L=N_n$ . A is the template that contains the weights given to the adjacent cell states (feed-forward) and B contains the weights given to the external input (feed-back). A and B define the rules to the memristor CA. Function $M(\cdot )$ describes the memristace change function (referring to Eqs. 1 and 2)¹⁸.

BPI training

The BPI training algorithm, proposed by Baldassi et al.¹⁷, is a bio-inspired learning model. Formally, under a network point of view, the BPI is applied to as a single layer of neurons with binary synapses able memorize input-output associations in a supervised way. The algorithm can be compared with the standard perceptron algorithm³⁹: training input patterns are given to the network one at a time, labelled with the proper output. In case of an error, the signal is back-propagated so that it can modify the synaptic weight values in a direction that makes it less likely to repeat the error in the future. Unlike the perceptron, the changes only affect an internal (hidden) variable in each synapse, which determines the synaptic weights (binary value)¹⁹. Formally, let us consider a network with size N (i.e. where there are N synapses) whose weight is $w_k$ and $k=1\dots N$. Assume also that the $w_k$ are binary valued, so that $w_k={0,1}$. Given the N input values $\zeta _k\in [0,1]$, the output of the network is computed as

$$\begin{aligned} \sigma ^{\chi }=\Theta \left( \sum _{k}^{N} w_{k} \zeta _{k}^{\chi }-\theta \right) , \end{aligned}$$

(4)

where $\Theta (\cdot )$ is the Heaviside function and $\theta$ is a given threshold for the current out-flowing from the system.

Let us train the network with a set of p patterns of N binary inputs $\zeta ^\chi \in \{0,1\}$, where $\chi \in \{1,\dots , p\}$, two kinds of outputs must be considered: the desired output $\sigma ^\chi _d \in \{0,1\}$, provided by the user and describes the association rule which the system needs to classify; and the real output $\sigma ^\chi \in \{0,1\}$, calculated for each input pattern. Aim of the training is the have the largest match possible between desired and real output. To do so, let us introduce a hidden variable $h_k$ associated to each synapsis, with $h_k\in [-1,1]$. For the sake of a simpler implementation, the $h_k$ may assume a limited number H of discrete values only. The relation between the $h_k$ and the $w_k$ is

$$\begin{aligned} w_{k}=\frac{1}{2}\left( {\text {sign}}\left( h_{k}\right) +1\right) . \end{aligned}$$

(5)

In order to train the network beforehand (i.e. update the resulting synaptic weights), at each given discrete time (or epoch) $\tau$, a pattern $\zeta ^{\chi }$ is chosen randomly from the training set associated with a given $\sigma _{d}^{\chi }$, in order to compute the stability parameter $\Delta ^\chi$ as:

$$\begin{aligned} \Delta =\left( 2 \sigma _{d}^{\chi }-1\right) (\sum _{k} w_{k}^{\chi } \zeta _{k}^{\chi }-\theta ). \end{aligned}$$

(6)

Depending on the value of $\Delta ^\chi$, the synapses in the network update according to the following rules:

• (Rule 1) if $\Delta ^\chi >1$, then $\forall k: h_{k}^{\chi +1}=h_{k}^{\chi }$ (i.e. do nothing);

• (Rule 2) if $\Delta ^\chi <0$, then $\forall k: h_{k}^{\chi +1}=h_{k}^{\chi }+\frac{1}{K} \zeta _{k}^{\chi }\left( 2 \sigma ^{\chi }-1\right)$;

• (Rule 3) if $0 \le \Delta ^\chi \le 1:$

• (a) if $\sigma ^{\chi }=0$, with probability $p_{s}$, if $w_{k}^{\chi }=0$, then $h_{k}^{\chi +1}=h_{k}^{\chi }-\frac{1}{N} \zeta _{k}^{\chi }$;

• (b) otherwise, do nothing.

Under a general point of view the hidden variables should assume a number of states $H=\sqrt{N}$ in order to achieve the best computational efficiency. The probability in rule can be set as $p_{s}\in \left[ 0,1\right]$, depending on the rule that the network must acquire. Computational experiments, showed that the tuning of these parameters is critical to achieve good performances¹⁹. In our proposed implementation, as it will be described in the following section, the hidden variables $h_{k}$ are stored as the memconductance of memristor devices which represent the cells of the complete system for wound analysis.

Memristor-based CA’s architecture with BPI algorithm for wound analysis

To find an equivalence between the Itoh and Chua’s general memristive CA model and its implementation of the BPI, several arrangements were made on the parameters of previously described models for the specific application. As afore mentioned a first generic convolutional neural network was implemented in the device in order to extract ROIs of wounds in the taken photos (see Fig. 1b)²⁴. This first network has been trained with wound images, so that the second network (the DT-CNN hereafter described) is able to perform its analysis just in the extracted ROI portion of the image. This mechanism allows to optimize the wound classification and obtain a general decrease of the computational costs and time. Regarding the architecture of the BPI-CA, as shown in Fig. 3a, our model was designed as three-layer cellular network, each layer taking into account one of the three digital channels of the image (i.e. R, G and B) and composed of a number of cells equal to the number of pixel in the ROI area. The cells take as input the brightness value of the corresponding pixel (i.e. with coordinates i, j) of the corresponding layer in an 8-bit digital form, so that $R_{i,j}\wedge G_{i,j}\wedge B_{i, j} \in \{1,2,...,256\}$, so that the input values of the three cells associated to the same pixel will correspond to the coordinates of a cell in tensor $\textbf{O}$ of size $256\times 256\times 256$.

Each cell (excluding of course that at the boundaries of the ROI) operates according to the same template, corresponding to the cellular automata considered above, that it is moved across all the ROI to analyse each pixel. A cell is locally connected to its corresponding input and also to the input of the eigth surrounding cells of the same layer, as well as to the two corresponding cells of the other two color layers, so that it has $N_n=11$ connections. Each connection is identified by a binary synapsis $w_{g,l,x}$, where $g,l\in \{-1,0,1\}$ identify the surrounding cell, and $x\in {R;G;B}$ identifying the color layer, simulating the generic binary memristor¹⁹ and has the scope of keeping record of the synaptic weights. To compute synaptic coefficients the network underwent a training session with the same dataset used to train the first sub-network (please refer to section Materials and Methods of this work for further detail) composed by over 1500 images of wounds, taken from open source databases²⁵ and manually segmented and classified. After training it was possible to identify a sub-space $\Omega \subset \textbf{O}$ in which are encoded the color combinations and their mathematical relations, that distinguish depicted chronic wounds from the rest of the background¹⁸. Moreover, it was possible to identify other sub-spaces $\Omega _1,\Omega _2,\dots ,\Omega _n \subset \Omega$ which distinguish features of different types of tissue composing the wound bed (i.e. necrotic, granular, fibrinotic, etc.), by labeling accordingly the information taken from the wound images in the training set. The wound identification from the picture is then performed by fixing the synaptic weights in O and moving the three-layered template across the ROI of a generic wound image, as shown in Fig. 3b generating a binary mask s whose elements are calculated as:The network provides a binary output for each pixel (i.e., the three cells associated to the three layers of a single pixel provide a single output) given by

$$\begin{aligned} s_{i,j}=\Theta \left( \sum _{g,l \in \{-1,0,1 \}} R_{i+g,j+l}w_{g,l,R}+G_{i+g,j+l}w_{g,l,G}+B_{i+g,j+l}w_{g,l,B}-\theta \right) ; \end{aligned}$$

(7)

$\theta$ is a constant threshold parameter that was accordingly tuned in order to filter eventual noise resulting from the image¹⁹. Concluding the elements of $s({i,j}=1$ correspond to the pixels that depict of the wound in the original image and are Therefore, highlighted. These are then classified regarding with respect to the identified sub-spaces of $\Omega \rightarrow \{\Omega _{1},\Omega _{2},...,\Omega _{n}\}$ and their proportions are counted in order to classify the wound according to the WBP score¹¹.

In order to train the network (i.e. properly identify the synaptic wights $w_{R,G,B}$ it is necessary to part from the equations and the rules describing the BPI. First, probability $p_{s}$ in Rule 3 was set to zero in order to have total control of the generation changes of the states of the system’s cells. Second, $h_{R,G,B} \in [-1,0,1]$, and represent internal state variable of the memristors composing ${\textbf {O}}$. It is necessary to note that the relation that links $h_{R,G,B}$ and the synaptic weights $w_{R,G,B}$ is described in Eq. (5) which also binarizes its final value according to the implementation described by Secco et al¹⁹. Therefore, it is possible to combine Rule 1 and Rule 2 with the definition of stability parameter $\Delta$, obtaining the evolution dynamics of a given cell $h_i$ in one of the cells of ${\textbf {O}}$:

$$\begin{aligned} h_{i}^{\chi +1}=h_{i}^{\chi }+2 \zeta _{i}^{\chi } \frac{({\text {sign}}(\Delta ^\chi )-1)}{2 {\text {sign}}(\sum _{k} w_{k} \zeta _{k}-\theta )}, \end{aligned}$$

(8)

where $\chi$ denotes the computational epoch. To adapt Eq. (8) to the designed network, we recall that the neighbourhood of each cell is composed by the triplets of pixel that correspond to the same position in the three layers of the template plus the first neighbouring cells in the same layer, as shown in Fig. 3a. The template is then moved across the ROI of a given image of the training set. To cope with this, we replace the hidden state $h_i$, $i=1,\dots ,N_n$ with $h_{g,l,x}$, with $g,l\in \{-1,0,1\}$ and $x\in \{R,G,B\}$. Also the input $zeta_i^\chi$, $i=1,\dots ,N_n$ at the epoch $\chi$ is replaced by the RBG values $\zeta _\{i+g,j+l,x\}$ of the image considered at epoch $\chi$, where (i, j) represent the pixel coordinates and ranges for all the identified ROI, and $g,l\in \{-1,0,1\}$ is the offset and $x\in \{R,G,B\}$ the layer. Therefore, Eq. (8) can be rewritten as:

$$\begin{aligned} h_{i,j,x}^{\chi +1}=M\left( A h_{i,j,x} ^{\chi }+\sum _{g,l \in \{-1,0,1 \} \wedge y \in \{R,G,B\} } 2\zeta _{i+g,j+l,y}^{\chi }\left( \frac{{\text {sign}}(\Delta ^\chi )-1}{2 {\text {sign}}(\sum _{k}^{N} w_{k} \zeta _{k}-\theta )}\right) \right) , \end{aligned}$$

(9)

with $x\in \{R,G,B\}$, (i, j) ranges to span all pixels in the ROI, and $\chi$ is incremented to consider all images in the training dataset, starting from the initial condition $h^\chi _{g,l,y}$ for all g, l, y. All the generation changes in the system must be input-based, so in the case of our CA-BPI, referring to Eq. (3), was found to be defined $A=1$ . It is possible to notice that the evolutionary algorithms described by the Eqs. (3) and the 9 present the same properties of a CA as described by Wolfram³⁸, thus are considered to be equivalent.

Fig. 3

(a) During the training of the BPI-CA, the three-layered template of nine cells per layer is moved across the ROIs channels of the wound images of the training set. At each epoch the template is moved by one pixel and takes in input the values of the of the R, G and B channels which serve as coordinates for the memristive cells in tensor ${\textbf {O}}$. (b) Similarly after training (i.e. when the value of the cells in tensor ${\textbf {O}}$ have been fixed) the template is moved across the ROI in which the wound is framed in the picture. Each layer of the template takes into account one of the digital channels of the photo and the single pixel values serve as coordinates of the given cell of the tensor. At this stage it is possible to recognize a sub-space $\Omega \in {\textbf {O}}$ in which are encoded the extracted color feature of the analysed wounds.

Results

Clinical trial

To prove the actual clinical reliability of the system a clinical trial on 150 patients, validated and approved on 20 September 2017 by the Ethical Committee of the Azienda Opedaliera Universitaria San Luigi Gonzaga (Orbassano, Italy), with protocol number OC15194 (see Fig. 4 for major details of the trial and its workflow) was performed.

The team was well-supported by the PI’s extensive experience, and the hospital chosen for centre of study is widely recognized as one of the leading departments in the care of complex chronic wounds.

Three cohorts based on their aetiology were considered: 1) lower limb ulcers (50 lesions), 2) diabetic foot ulcers (50 lesions), 3) pressure ulcers (50 lesions). The sample size has been calculated by evaluating a statistically significant number of samples for each aetiology (at least 50). The choice of the cohorts and its organization was made to reflect, in the best possible way, the broadness of occurrence of cases in a specialized wound care centre^1,40. Regarding the population demographics, the participants had to match the inclusion criteria:

• 1. The wound must be considered chronic.

• 2. Absence of subtypes.

• 3. Willing of signing of the informed consent.

• 4. The area of the lesion must be between $2 cm^2$ and $100 cm^2$.

It is also important to consider the skin tone variable of the patients that could introduce a bias on the wounds classification. In addition, as mentioned previously, the data set used for the network’s training included images of wounds in a broad range of skin tones returning high validation rates. Moreover, the skin tone rarely matches with the deeper classified wounds’ tissues colors, Therefore, the patient’s skin characteristics were not inserted as an inclusion or exclusion criteria. The study was designed as a prospective and observational. The trial is non-randomized and mono-centric with the blinding assessment of the type of wound for the physicians²³.

Fig. 4

Workflow of the clinical trial performed at the San Luca and Politecnico di Torino. As shown, the trial involved two teams: a clinical team and a data management team which worked in double blind. The clinical team had the responsibility of enrolling the patient, perform an analysis with the WV device and other manual measurement devices for wound care. In addition, the clinical team had to perform a visual assessment of the wound, classifying it through the WBP score. All the data obtained with the devices (excluding the WV) were reported on paper Clinical Research Folders (CRFs), while the data obtained with the WV was automatically sent to the data management team through a secure cloud data transmission system. The Data Management team, at the end of the trial, collected all the CRFs and compared the obtained results calculating the outcomes.

Wound classification method performance

The results obtained have been reached with the totality of the patients recruited in the clinical trial. The analysis was based on the ability to identify the tissues composing the wound bed in the proper percentage and to relate these with the WBP score using the techniques described in the previous paragraphs of this work with the WV. Through WBP, wounds can be subdivided in four classes (A, B, C, and D) according to the description provided in the previous sections of this work¹¹. This evaluation allows the ability to correlate the color segmentation of the wound bed against the correct WBP score and to compare its result with the one given by the clinical assessment team who examined the wound visually as described in Fig. 4. The aim of the protocol was designed not only to verify the classification accuracy of the device, but also to prove that the AI medical device allows trained physicians to gather precise, reliable clinical information in a telemedical configuration, as if they were performing the same examination directly on the patient²³.

The devices’ WBP scores were compared with those obtained by physicians during clinical visual assessment.

From the point of view of the WBP classification performance of the BPI-CA algorithm, the accuracy results are shown in Fig. 5a. As shown, a weighted average that takes into account the numerosity of the singular cases, of 96% of cases the two assessments (i.e. the visual performed by the clinical team and the one performed by the WV) coincided was calculated. . More specifically, the confusion matrix reveals on the diagonal the number of cases in which the classifications matched. On the other hand, the other points outside the diagonal count the cases where there was a mismatch in the wound classification. The mismatches were registered in three cases related to C and D class. The inter-rater agreement has also been evaluated through the Cohen’s Kappa methodology, between the WV and visual assessment, obtaining an optimal value of 0.9728, that confirmed high statistical agreement of the obtained results.

Fig. 5

(a) Wound classification and accuracy performances described through a confusion matrix. (b) Four examples of wounds assessed during the trial in terms of WBP score, for the assessment performed by physicians and AI medical device. In the end, the results of the color analysis performed by the AI medical device algorithm. (c) Three examples of wounds analysed during the trial. All three wounds were classified both by the physician and by the AI medical devicer as D (presenting eschar or necrotic tissue).

As an example, four cases that were assessed during the trial, are reported, one for each WBP class. In Fig. 5b is shown a series of wounds where each proper class has been identified both with visual and AI’s medical device and it is reported also the tissues component percentage in terms of color detected by the AI’s algorithm. As shown, it can be noted that wounds with A class reported 100% identified granulating tissue. B wound is characterized by a granulated tissue percentage in range 50–100%, in fact granulating tissue is reported as 82%. C wounds have less than 50% of granulated tissue, so in the case reported in the figure the granulating tissue percentage is 45%. Finally, the D wounds are necrotic (even slightly) and the reported case was found to be composed of 38% eschar.

The precision of this detection is clinically fundamental fundamental, mostly detecting cases which present necrosis. As shown in Fig. 5c, the assessment for D wounds is comparable between visual and the WV. In figure it has also been reported the identified necrotic percentage of the shown wounds’ beds. By the end of the trial around 50 wounds were classified as ‘D’ (according to the WBP scale), meaning that these wounds presented necrotic tissue, even at a low percentage. ‘D’ classified wounds require more attention by health professionals because of their highly advanced state of deterioration. The results of the analysis showed that the AI medical device was able to recognise these particular wounds with high precision, proving at first that the objective for which the device was designed was reached.

Morphological measurement precision

The morphological measurements precision has been evaluated by comparing the data obtained from the WV with the corresponding measurements collected by three other wound analysis and measurement methods²³:

• VISITRAK (Smith+Nephew, US) is a wound measurement active board formerly distributed by Smith and Nephew. The device comes with a tracing pen and a set of scaled transparent acetate sheets.

• MOWA (Health Path, Italy) is an app for Apple IOS mobile platforms.

• Aranz Silhouette (Aranz Medical, New Zealand) system is composed of the Star, an image acquisition tool comprising three laser arrays, a digital camera and two LEDs for image light control.

Table 1 Mean percentage measurement relative error with the respective standard deviation between the WV and the other devices used to evaluate the WV measurement precision capabilities precision, regarding wound areas in $[\hbox {cm}^{2}]$.

The results, in terms of mean error percentage of the measurements performed with WV compared to the ones performed with the other devices, are reported in Table 1. To perform a more relevant analysis regarding the morphological measurement precision, the wounds analysed with the four devices, were divided in different size classes. For each size class. It is interesting to note how WV, Silhouette and VISITRAK have smaller mean relative errors, whereas MOWA seems to be less reliable, with mean relative errors reaching over 10%. The highest errors were obtained measuring wounds of 16.8–22.4 $\hbox {cm}^{2}$. In these particular cases, the results were accepted since the differences in terms of relative error are clinically irrelevant. These errors were computed by considering the three other wound analysis and measurement methods as imperfect controls.

Table 2 Shape and scale parameters of the overlapping Weibull distributions of the distributions of the collected data. In the last row are reported the p-values of the Ks-test that was performed to confirm the correct fitting of the curves to the population.

In order to find further statistical evidence of the measurement capabilities of the system, the data from the AI medical device and control devices were compared by creating distribution curves for area and depth measurements respectively. Such distributions resulted to be comparable to Wiebull curves. Therefore, the scale and shape factors of these curves were used to generate fitting distribution curves, and their goodness-of-fit was evaluated through Kolmogorov-Smirnov test (Ks-test). The estimated parameters and corresponding values are presented in Table 2. The Kruskal-Wallis test, with a p-value acceptance threshold of $\ge 0.1$, was employed to determine if the distributions were statistically comparable. This non-parametric test was used in place of the one-way ANOVA for non-normal populations. The test yielded a p-value of 0.9, which affirms that the distributions represent the same population. From these results it is possible to state that the BPI-Ca was able to perform reliable and precise measurements, with high clinical relevance for a normalized clinical use of this technology.

Clinical and economical advantages

Following the afore described trial, the WV has been integrated in different hospital facilities. A further evaluation comparing the data regarding the cost of cures between the year before the use of WV and the following in one particular hospital was performed to show the positive impact of such technologies in the wound care field with respect to the general health administration. Table 3 reports the data obtained through this analysis in terms of cost of cure. These costs are also divided into the three major expenditures that are: the medication costs (i.e. the cost of the prescribed dressings), the visit costs (i.e. the cost of the single specialist traveling through the territory to perform medication) and the cost for the therapeutic plan (i.e. the administrative costs for medical prescription). Surprisingly, thanks to the use of WV and its capability transmit the information among the operators directly into the patient’s Electronic Medical Record (EMR), the wound care specialists were able to administer the right therapy to the single patients according to their general clinical state. Moreover, it was possible to render efficient the general operations regarding the patient management, concentrating the operators on the patients that required greater treatment and attention, lowering the number of visits to the ones that were going through a correct healing process. From these logical actions taken by the hospital through the use of WV, the total cost reduction for the hospital (on a population of around 850 patients per year) has been reduced by 9%, while the cost of cure per single patient reduced by 14%.

Table 3 Cost of cure reduction (total and per single patient) in telemedical procedures regarding patients with chronic wounds.

As mentioned previously the presented analysis was performed over a one year period. The scenario represents an example of normal wound care territorial management facility, subject to a very high turnover of patients due to healing or dramatic worsening of their health conditions⁴¹. Therefore, it is possible to state that though the analysis involves a short time period, it can be considered sufficient for highlighting the short term improvements not only for the patients, but also for the wound care personnel and their management. Notwithstanding, further observations for longer periods of time that include the comparison of more facilities afferent to different health care systems would certainly result in additional social and economic outcomes that are difficult to assume a priori. For this reason, additional analysis is being conducted following novel implementation of the WV in different environments of cure.

Discussion

The novel WV device that implements the BPI memristor-based CA classification model has been used to detect and analyse a cutaneous ulcer from the digital picture of a patient’s lesion. Technologies for automatic diagnosis using a neuromorphic approach are still largely unexplored and not well-documented in the literature for wound classification purposes, except for few notable cases that were able to reach actual clinical use. For instance, in 2017, the MolecuLight Imaging Device was introduced as a portable, non-invasive, real-time camera for bacterial proliferation in wounds using UV light illumination and a dual-bandpass optical filter to capture bacterial fluorescence⁴². Other systems have also emerged that exploit AI technology for wound identification and measurement, but on the other hand lack of diagnostic classification features and are not as widely distributed in order to perform a complete comparative assessment^43,44,45. On the other hand, the WV represents an innovative and unique certified medical device that has been clinically tested, able to perform both automatic wound measurement and clinical classification through validated and widely used clinical scales such as WBP.

For its validation Therefore, the choice fell on the range devices most commonly used by wound care personnel, which represent the actual clinical operative standard in terms of wound morphological measurement. Notably, none of these gold standard devices employs a neuromorphic algorithm for ulcer type diagnosis. Consequently, its diagnostic validation was performed through a double blind comparison with wound care experts’ visual assessment.

The results have demonstrated good accuracy and performance in terms of classification regarding the WBP score of the wounds.

In particular, the visual assessment obtained by the physicians matches almost perfectly with the WV. Most importantly the classification capabilities of the device resulted to be highly specific in cases where wounds present necrosis. To prove such capabilities, the wounds shown in Fig. 5a present a very low amount of necrotic tissue which was correctly detected by the device. Moreover, the described system and the trial it underwent, faced a challenge related to performance across different skin tones and under varying lighting conditions. It must be noted that the trial was conducted in an operative wound care ward with variable light conditions and the patient enrolment randomly included patients with different skin tones (also including 6 subjects with African traits and 4 subjects with Asian traits, Males: 36%, Females: 64%). These variables, given the obtained results, did not affect the overall performance of the system. This positive outcome can be conducted to the implemented illumination system of the device through its 4 mounted LEDs and a correct choice of the samples that populated the training dataset.

The BPI-CA was found in general to be able to correctly detect the contours of a wound: this algorithm showed very high precision with a fully automated process of extraction and detection of the wounds and their characteristics, without any interventions from the examiners. All the BPI-CA results obtained used in the clinical trial, showed a strong consistency compared with the ones from other devices, with the difference that the BPI-CA algorithm doesn’t need any manually operation by the physicians, except the picture of the lesion, as added value.

The final analysis addresses the actual scope of precisely assessing wounds. To arrange the proper medications and treatment guidelines, it is vital to assess the correct wound class. With the recent advancement of artificial intelligence, such as the WV, wound specialists can classify wounds in better accordance with all the physiological parameters and aspects of the patient and do not only through isolated wounds image. These possibilities and advancements reflect the positive trend taken by certain hospital facilities in terms of general a single patients’ expenditures, as shown in this work. The costs in terms of time, medical visits and treatment may increase sensibly in absence of a of appropriate clinical assessment tools in the field of wound care. The clinically validated WV has proved able to render a positive impact for physicians in patient management and treatment.

Limitations

As detailed in the Clinical Trial section, patients were divided into three main groups based on their wound aetiology. The group with lower limb wounds included patients with various aetiologies: approximately 40% had venous ulcers, around 20% had arterial ulcers, and the remaining 40% had ulcers of other origins. This group does not encompass the full spectrum of possible aetiologies. Ongoing research and studies are focused on conducting trials specifically targeting different wound aetiologies to evaluate the clinical benefits, with patients receiving regular follow-ups. Additionally, patients with wounds exhibiting undermining were excluded from the trial due to technical limitations of the device. Efforts are underway to enhance the system’s capability to detect these conditions²³.

As afore mentioned, the described trial included a relatively small amount of subjects with different skin characteristics with respect to the overall average patient population affluent to the trial’s centre. Though the analysis of the wounds of this population did not compromise the system’s performance, their limited number is still to be considered statistically irrelevant, and further studies are ongoing in order to assert the lack of impact of such variability. Although the presented dataset was heterogeneous in skin tone variability, future investigation on the potential source of bias introduced by skin tone variability will be properly evaluated. It is necessary to note that in a clinical environment lesions and the skin area them are not clean and present debris from previous dressings, ointments and bandages, that can also be present inside the wound. This variability is independent from the patient and constituted a higher challenge during the network’s training rather than skin tone, which in general has features that are very different from wound bed tissue. By correctly segmenting and classifying the images of the training set the device was able to overcome such problem as the presented trial results suggest.

Conclusion

Finally, the BPI-CA algorithm shows promise as an approach in wound care, with initial results demonstrating good efficacy within the limitations of this study. Although the findings are quite limited and need for broader and long-term studies, the goal of improving clinical care, and reducing costs in all medical fields, meets the AI technologies’ development, finding the optimum point in addressing an urgent clinical need within dermatology and a necessary standard protocol for quality of care^6,46,47. As mentioned, the classification and measurement capabilities of the WV and the possibility to transmit relevant data makes the device an efficient telemedical tool for the remote monitoring of hard-to-heal wounds. The recent pandemic event has forced the great part of the national health systems to improve this kind of standardised protocols for remote patient care, without compromising the quality of clinical standards. In conclusion, the effort required by operators can be decreased by the use of this device, focusing the work of physicians on advanced treatment and more efficient patients remote care, leaving room for artificial intelligence systems to manage data and give reliable support in the diagnostic assessment.