Introduction

Systemic Lupus Erythematosus (SLE) is a systemic rheumatic disease with multiorgan involvement that takes a long time to diagnose and, therefore, causes delays in treatment1,2. The diagnostic process of pediatric SLE (pSLE) and all childhood rheumatologic diseases may take time due to the need for more objective findings and clinician dependence1,3. In recent years, Artificial Intelligence (AI)—based systems have been developed to support clinicians in many areas4,5. One of the most important reasons for this is the low number of pediatric rheumatologists (PRs) compared to the population6,7,8. On the other hand, clinics that treat urinary tract infections and skin diseases, which are common in the pediatric age group, can often be confused with pSLE6.

Another problem is that patients cannot be followed up in the triage system due to a lack of records. Even if patients reach PRs, the diagnostic process may take months because there are no specific diagnostic criteria for childhood SLE patients9. When all these problems are evaluated, the pSLE diagnosis process should be digitalized, and a model free of existing issues should be developed4,10. We aim to establish an AI-supported model that helps PRs and physicians in all fields diagnose pSLE.

Materials and methods

Study group and design

Fifty pSLE patients followed up in Hacettepe University Pediatric Rheumatology Outpatient Clinics, and 50 healthy individuals similar to them in terms of age, gender, and sociocultural characteristics were included in the study. Clinical data of the patients were obtained from the hospital automation system. The healthy control group was selected from the patients’ peers (usually schoolmates or peers living in the same neighborhood). Since the clinical and laboratory data of lupus patients were evaluated at the time of diagnosis, blood samples were not taken from the patients and were not examined. The information of the individuals participating in the study was recorded in an offline computer-based data set. Access to the dataset was restricted except for those conducting the survey.

The study used fully anonymized pSLE patient data previously used by the Department of Pediatric Rheumatology of Hacettepe University Faculty of Medicine and had official approval from the relevant department. Written and verbal consent was obtained from all patients for these previously used data for different studies. For this purpose, patients were invited to Hacettepe University Faculty of Medicine, Department of Pediatric Rheumatology, for one-on-one information, and their signatures were obtained on the 'informed consent form’ after verbal and written information was given. For participants who did not come to the department in person, the relevant consent files were sent to their e-mails after verbal information and approval by telephone, and their data were used if the signed files were scanned and delivered to us. All study methods were carried out following the relevant guidelines and regulations. In addition, an approval from the Ethics Committee of Hacettepe University Health Sciences Ethics Committee dated 09.24.2024 and numbered E-24742385-000-00003782004 was obtained.

The study design was planned to be double-blind. Thirty mixed-diagnosis cases (data entirely foreign for the model and the clinician) were submitted to the AI model, a concurrent expert pediatric rheumatologist (PR) and their interpretations were compared. The PR whose responses were compared with the AI model was utterly unfamiliar with the study and the data presented. Of these data, 15 were pSLE cases, 10 were healthy controls, and three were followed-up individuals with only anti-nuclear antibody (ANA) and anti-double stranded DNA (anti-ds-DNA) positivity and no other clinic. One was a newly diagnosed, untreated individual with acne rosacea who developed a skin infection due to secondary Staphylococcus epidermidis. Another patient had a urinary tract infection due to an adenovirus infection.

AI-based closed model

The reason for choosing the AI Closed Model is the law on 'Protection of Personal Data’ in force in Turkey11. According to this law, data processing and evaluation cannot be shared with third parties, institutions, and organizations12. Online AI-based applications or correspondence sites cannot be used to process sensitive data such as patient information. Therefore, although the model created has autonomous features, it can never be processed in online systems. Python software and Tensorflow were preferred for model development due to their ease of use in medicine and healthcare today13,14. We also chose these resources for developing the model because they have been used in medicine and many other data-processing fields for years with almost negligible error rates. The model was named ‘Morgaf’ by Dr. Emil Aliyev, M.D., M.S. (hereafter, the AI model will be referred to as Morgaf). With this name, an application has been made to the Turkish Patent and Trademark Office, and the patenting process is ongoing15.

The concept of a ‘Neural Network’ (NN) is becoming increasingly widespread in AI studies14. NNs consist of neurons and are structures that calculate and produce output with a predetermined mathematical function. In the development of Morgaf, the 'Recurrent Neural Network’ (RNN) model was preferred (Fig. 1)16.

Fig. 1
figure 1

Schematic description of Morgaf’s RNN model. SLE: Systemic Lupus Erythematosus, HC: Healthy Control, RNN: Recurrent Neuron Network.

Full size image

The most important difference between the RNN and other networks is that while training the model and processing the inputs at Tn, it also remembers the inputs at Tn-1 (Fig. 2) (Tn is any time unit).

Fig. 2
figure 2

Graphical description of the data processing interval of Morgaf’s RNN in units of time. Tanh; hyperbolic tangent function of any unit of the time.

Full size image

This feature makes the error rates on the network much lower than other models. The laboratory and clinical results of the patients were digitized as 1 (abnormal), 0 (normal range), and − 1 (abnormal). Data sets containing clinical and laboratory findings of individuals at the time of diagnosis were given as input to Morgaf. Morgaf is designed as an RNN model that constructs an NN with 1024 neurons (Fig. 1). The model needs many inputs to ‘learn’ the data. A total of 100 data sets is generally insufficient to build a closed-loop AI model.

For this reason, the 100 data sets were increased 15 times to 1500 with repetitions, 80% of which were reserved as training data and 20% as validation data. Training data is the data that Morgaf continuously ‘repeats’ during the learning process to ‘train’ itself. Validation data is the data that Morgaf checks itself after momentary training (Fig. 3).

Fig. 3
figure 3

Schematization of Morgaf’s process of teaching itself from limited data. pSLE: Pediatric Systemic Lupus Erythematosus, HCs: Healthy Controls.

Full size image

In the early training phase of Morgaf’s training, the error rate was approximately 0.5. In the later generations of the model, the error decreased to 0.028. This error rate was deemed sufficient, and Morgaf’s training phase was terminated. Training continued autonomously in the latent phase. Autonomous training in RNN models such as Morgaf is essential for continuous self-evaluation and error outputs.

It is essential to interpret the outputs to understand the ‘language’ of Morgaf. According to the output Morgaf at the end of the training, a diagnosis was predicted to be between 0 and 100%. Accordingly, it was seen that Morgaf determined the output as 70% and above to say definitive diagnosis pSLE, 10% and below to say healthy individual, and 10–70% in cases that require follow-up and in between (definitive pSLE and not healthy, but need to be followed up). Thus, the beta version of Morgaf was made ready.

Data redundancy was avoided so that Morgaf could most accurately recognize pSLE. Based on the ‘mental algorithms’ of clinicians when diagnosing pSLE, Morgaf was taught the clinical and laboratory data that are key to the diagnosis (Table 1). For example, instead of the amount of protein in the urine, the clinician was taught whether it was present or absent, and instead of the titer of ANA. The clinician was taught whether it was positive or negative. As a matter of fact, in practice, clinicians continue their diagnostic algorithms by considering an ANA value above 1/40 as positive17. This is one of the most critical steps model builders take when developing an AI model. Because it is not the number of data given for a single patient, making it possible to reach the target with fewer data is the key to the model’s success. In this context, the most important clinical and laboratory data of pSLE patients at diagnosis were used to create the model (Table 1).

Table 1 pSLE patient data used in the development and training of Morgaf and its digitalization.
Full size table

In addition to Morgaf and PR having a 100% case recognition rate, their median responses for each case ranged from 0 to 100%. For example, when predicting a patient with SLE, Morgaf said 83.21% for this patient. Since it was in the range of 70–100%, he correctly recognized the SLE patient. Since the clinician was answering whether the patient had SLE or not, we expected the clinician to express a percentage value for each case, so unlike Morgaf, his answers ranged from 0 to 100%. However, because it correctly predicted all SLE patients, the accuracy percentage was 100%. Another critical step is quantifying these data in a way Morgaf can learn. The inputs were then presented to Morgaf.

Statistical analysis

The study data were computerized and analyzed using the Statistical Package for Social Sciences for Windows version 22.0 (SPSS Inc, Chicago, IL)18. Visual (histograms and probability plots) and analytical methods (Kolmogorov–Smirnov/Shapiro–Wilk Test) showed that the variables measured in the study did not fit a normal distribution. Mann–Whitney U test was used to compare the variables between two independent groups. The statistical significance level was accepted as p < 0.05. Levene’s correlation analysis tested for homogeneity of variances and non-normally distributed variables, and variables with a correlation coefficient < 0.1 were considered uncorrelated. Prediction success was evaluated by performing regression analyses between both groups.

Results

Morgaf accepted 70% and above as a definitive diagnosis of pSLE, and the mean percentage of diagnosis was 93% (78–98%). Morgaf received 10% and below to call it a completely healthy case; the average was 1% (0–3%). Morgaf assigned himself a range of 10–70% to recognize diseases requiring follow-up; the average was 33% (15–47%). Morgaf correctly recognized all cases of pSLE, healthy individuals, and definite non-pSLE cases (Table 2).

Table 2 Case recognition boundaries and results of the Model’s training.
Full size table

There was no difference between Morgaf’s estimates and actual diagnoses (p = 0.297). It was 100% successful in recognizing lupus disease. This rate was the same as the diagnostic success rate of clinicians. Morgaf (93.3%) gave more precise recommendations for non-lupus cases than clinicians (70%) (p = 0.034) (Table 3).

Table 3 Comparison of Morgaf and clinician responses.
Full size table

Covariance analysis was performed to test the homogeneity of the variables (p = 0.792). Linear regression analysis was performed for both homogeneously distributed groups and tested with Levene (p = 0.147). Regression analysis showed that Morgaf (y = 0.926xi) was more successful in predicting findings than clinicians (y = 0.7322xi).

Discussion

AI-based models have been used in many areas of medicine19. AI models are increasingly invested in medicine as time goes by20. Although AI and deep learning models have been developed in medicine, unlike other fields, these models have not been implemented4. Especially in imaging, tools that evaluate radiological images through U-net software and help clinicians are familiar21. AI models that analyze and predict magnetic resonance imaging, direct radiography, and ultrasound images are standard22,23,24,25,26. However, all of these are applied in clinical practice in the clinics where the study was conducted, and they are not widely used19,26. The main obstacle to this is the legal barriers to sharing data with other clinics and that the data are generally online-based AI models27. As the name suggests, online AI models can use all online data28,29. Typically, applications that help diagnose symptom inputs are based on these models29. However, we know that patients and their complaints are individual. If the answers are not obtained from reliable sources or cannot be monitored, these applications may be biased and raise ethical issues such as violating personal data30. Therefore, our model is an offline model. This aligns with the data protection principle31. Apart from this, our model does not process data from online databases or online sources when making comments. The model can only work on the data we provide. That is, it can comment on the data of pSLE patients.

Apart from AI models that analyze and interpret images, the second most frequently used area is the analysis of large data sets and models based on prediction development24,26. This is natural because we know that AI models require a lot of data in terms of data processing. However, this is a speed-limiting step in areas such as rheumatology, where the number of patients and data are scarce32. When we developed Morgaf, we tried to solve the problem of how to train a model from limited data. We replicated the data by fully randomized and subsequently validated by the clinician to verify that they were similar to accurate patient data. Of course, this can be considered a limitation of the model since we created synthetic data. Still, it should be remembered that data scarcity is a severe problem in most fields of medicine, and it is essential to prevent this. This interpretation of Morgaf makes it unique. Of course, we are aware that more real patient data would train the model better. Therefore, larger data sets and prospectively designed studies are needed.

Another research topic in rheumatology is applications that lead from symptom to diagnosis29,33. Since these models are also online, they cause severe ethical problems due to violating personal data and biased interpretation30. Morgaf, the model we developed, is an offline model.

Interestingly, Morgaf, when comparing thirty completely unfamiliar data sets with the clinician’s interpretations simultaneously, recognized non-pSLE cases (Fig. 1), unlike the clinician, even though we had yet to teach it to do so. This showed that Morgaf could easily differentiate between other clinics as it had ‘learned’ the data of pSLE and healthy individuals, so much so that it could make better comments than the clinician. It is possible to ‘teach’ Morgaf visual data, such as pathology preparations of pSLE patients, in the future so that it can be a more successful model. However, at this stage, we preferred to use a single programming language and library as it would reduce the model’s chances of success when processing complex data. This can be considered a limitation of our model in fully recognizing pSLE disease. This model also needs to be tested in multiethnic cohorts and in other centers.

It is worth noting that Morgaf was well acquainted with childhood-onset SLE patients but not with other differential diagnoses. Furthermore, the model has not been tested in adult SLE patients. This was important to create the contrast between intact control and SLE that we needed when training the model. However, providing specific clinical and laboratory data on autoimmune, hematologic, infectious and many other diseases with differential diagnosis of SLE during the training of the model could enable the model to become a robot that more sharply recognizes SLE in real time. This is an important limitation of our study.

Morgaf is the first AI model in the literature to be developed from limited data and successful in pediatric rheumatology and pSLE disease. The model is under development and is expected to be successful as a cost-effective model. If integrated into hospital automation systems (Fig. 4), it is predicted that the model may lead to earlier diagnosis of pSLE patients, including non-specialty clinicians, significantly reducing hospital costs.

Fig. 4
figure 4

Schematic description of Morgaf’s functioning as an AI model.

Full size image