Background & Summary

Branch retinal vein occlusion is recognized as one of the most prevalent sight-threatening retinal vascular diseases observed in adult cohort1,2,3. The reported prevalence of BRVO is estimated to be 0.4% worldwide with no gender difference3,4. As the growth of the age increased risk, and patients with monocular BRVO have a 10% risk of developing retinal vein occlusion (RVO) in the contralateral eye within a three-year period5. The pathogenesis of BRVO is multifactorial and has not been fully determined6. BRVO can lead to macular edema, insufficient blood supply to the retina, and thus blurred or impaired vision7. In some cases, BRVO may cause retinal oxygen deprivation, which stimulates the formation of potentially new blood vessels8.Theses vessels are fragile and prone to rupturing and bleeding, further damaging vision9,10. BRVO may also lead to complications such as glaucoma, and vitreous hemorrhage. An example of the different imaging modes is shown in Fig. 111. Therefore, although it may be asymptomatic in the early stages, can have serious effects on vision if left untreated12. The clinical treatment of BRVO patients involves two practical aspects: firstly, reducing macular edema and improving visual acuity; secondly, minimizing the occurrence of complications13. Currently, there are three anti-VEGF drugs available: ConberCept, Lucentis, and Eylea. Large clinical studies have demonstrated that all three drugs yield positive outcomes, with over 50% of patients experiencing significant vision improvement14,15.

Fig. 1
figure 1

(a) Retinal structure; (b) (c) Ultra-wide-angle fundus photographic image of patients with BRVO disease; (d) OCTA vessel image of macular edema secondary to BRVO; (e) OCT structure image of macular edema secondary to BRVO.

Full size image

Fluorescein Angiography (FA) helps characterize the retinal vascular system, including degree of non-perfusion, macular ischemia, macular edema, and leakage16. Although FA currently serves as the gold standard method for clinical evaluation of retinal perfusion, problems such as internal retinal bleeding, invasive nature and time-consuming inability to be repeated at each visit, as well as its incapability to visualize retinal capillary plexuses at different depths and the potential adverse effects associated with intravenous dye injection can make it difficult to obtain high-quality FA8,17,18. In contrast, OCTA is a recently developed non-invasive technique that utilizes a split-spectrum amplitude decorrelation vascular imaging algorithm16,19. It provides both structural and vascular information, enabling imaging of the superficial layer, deep layer, outer Retina layer, and choroidalcapillary layer of vessels, respectively20. Several studies investigating OCTA in BRVO patients have reported microvascular abnormalities such as telangiectasias, microaneurysms, collateral vessels, and nonperfusion zones20,21. Furthermore, these studies have demonstrated that OCTA can depict abnormal vascular changes equally or even more effectively than FA21,22,23.

Given the recent advancements in artificial intelligence (AI) technology in computer-aided disease diagnosis and treatment, multiple significant advancements started to focus on applying AI methods for automated analysis of BRVO. To foster the development of OCTA image analysis and processing technology, Nanjing University Of Science And Technology and Cixi Institute of Biomedical Engineering has publicly released OCTA-50024,25 and Rose26, respectively. However, these datasets were collected from a single visit to various patients and encompassed a wide range of eye diseases. Furthermore, the creation of these labels heavily relied on the manual efforts of domain experts, which consumed significant amounts of time, energy, and financial resources. To the best of our knowledge, prior to this study, there was no publicly available OCTA database specifically focused on a particular ophthalmic disease with multiple diagnostic capabilities. In order to foster research on BRVO disease, we have released Soul dataset comprising image-text data from 53 patients. This study was received approval from the Institutional Review Board at the Affiliated hospital of Shandong Second Medical University (wyfy-2020-ky-11), following the principles outlined in the Declaration of Helsinki and with the include informed consent of all participants.

Finally, our contributions include:

  1. (1)

    The datasets we constructed consist solely of patients with BRVO, all of whom have been diagnosed with BRVO disease due to vascular loss or macular edema, resulting in various changes in image data. These datasets do not include normal disease-free eye imaging data.

  2. (2)

    Multiple pre-operative and post-operative follow-up visits were performed and the data were divided into different subsets. This allows for longitudinal analysis of patient changes and expands beyond a single segmentation task. Researchers can now select appropriate data for clinical research purposes such as disease diagnosis and prognosis after completing the segmentation task.

  3. (3)

    We have developed a comprehensive framework for automatic vascular label formation that utilizes model integration technology and deep learning knowledge to generate high-quality vascular labels. In comparison to alternative approaches, this framework not only attains a high level of accuracy but also significantly mitigates the expenses associated with manual annotation.

Methods 

We categorize the generation of Soul datasets into three primary stages: initial clinical data acquisition, subsequent data filtering and pre-processing, and ultimately the implementation of a Human machine collaborative annotation framework (HMCAF). The overall workflow is illustrated in Fig. 2.

Fig. 2
figure 2

Overall framework of Human machine collaborative annotation framework (HMCAF).

Full size image

Clinical data acquisition

Soul is a multi-diagnosis dataset collected from 53 subjects. The data collected at baseline include OCTA retinal images among BRVO patients in ages between 31 and 82 years old retrospectively collected from the Affiliated hospital of Shandong Second Medical University between 2020 to 2021. This study was received approval from the Institutional Review Board at the Affiliated hospital of Shandong Second Medical University (wyfy-2020-ky-11), following the principles outlined in the Declaration of Helsinki and with the include informed consent of all participants. OCTA images were included if (1) They do not have noticeable quality issues, such as severe smudges, artifacts, out-of-focus, blurriness, incorrect exposure, etc., that would affect the clarity of the observed target area. (2) To ensure spatial alignment of longitudinal tracking data, the macular fovea area is first labeled by a professional ophthalmologist during each patient visit. Subsequently, the 6*6mm area of the fovea of the macula is identified by automatic spatial alignment using the Optovue Angio OCT RTVueXR, a specialized ophthalmic device. This process uses high-resolution optical coherence tomography angiography to ensure accurate alignment of data from each image, helping doctors better monitor the progression of lesions and develop more effective treatment plans. An example is shown in Fig. 3, The results of multiple imaging data comparison showed that the blood vessels were reconstructed to a certain extent after multiple treatments. (3) Since the anti-VEGF Ranibizumab Injections drug is only valid for one month, in order to ensure the validity of follow-up data, only the information of visits on the day of Injections and visits within one week of Injections are kept in the follow-up data, and the follow-up should be ensured within 2-3 days after surgery as far as possible. Images were excluded if they showed any evidence of treatment, severe exposure abnormalities, severe refractive interstitial opacities, large-scale contaminations or if information about its origin was missing.

Fig. 3
figure 3

The spatially aligned OCTA image data of BRVO patients corresponded to surgery 1, surgery 2 and surgery 3, respectively.

Full size image

Text Record

The data primarily encompasses diverse medical information of patients, encompassing collection numbers, gender, age, diseased eye (left and right), disease progression, surgical dates, follow-up visit count, macular center thickness measurements, visual acuity assessments etc., with the corresponding records stored in an Excel file. To ensure participant anonymity and confidentiality, personal identifiers such as names are removed which can identify the subjects identity.

Image

The OCTA images typically acquired and stored grayscale in nature, saved in JPG format. The scanning process employed the Optovue Angio Oct RTVueXR system. It took approximately three years to collect and annotate these images. All subjects have complete registration information, with diagnosed diseases provided by ophthalmologists. Figure 4 (left) illustrates an sample of the scanned images alongside Projection maps at various levels. Although OCTA can produce projection Superificial layer (SVC), Deep layer (DVC), Outer Retina layer, and Choriocapillaris layer according to different retinal projection map, since BRVO disease image features are mostly based on the Superificial layer information, we have only take the SVC layer as the research object and build labels for its images. The retinal hierarchy corresponding to different projections is shown in Table 1 and Fig. 4 (right).

Table 1 Superificial layer (SVC), Deep layer (DVC), Outer Retina layer, and Choriocapillaris layer corresponding retinal layers.
Full size table
Fig. 4
figure 4

Original scanned images(left) and Projection maps at different levels(right).

Full size image

Model Pre-training and Data pre-processing

Soul encompasses three subsets, which have been categorized based on the number of injections and follow-up periods. These subsets include Soul-1(s1t1 & s1t2), Soul-2(s2t1 & s2t2), and Soul-3(s3t1 & s3t2), corresponding to patients who underwent a minimum of one, two, and three surgical treatments respectively. The characteristic of HMCAF is to reduce the time investment of expert manual annotation as much as possible while ensuring the authenticity and accuracy of labels. To ensure image quality and model performance, before original images imported into the baseline models of the HMCAF framework, A series of data processing operations such as data normalization, scale transformation, brightness change and contrast change were carried out. All images were uniformly cropped to eliminate any unused or unimportant boundaries and resized images with 304 * 304 pixels samed as OCTA-500 and ROSE. Before generating initial labels, the baseline model in the HMCAF framework needs to be pre-trained on the OCTA dataset to produce more accurate results. We pre-trained each of the four baseline models on the open source ROSE dataset and achieve optimal performance on the ROSE dataset. The baseline model performance is shown in Table 2.

Table 2 Performance of the baseline model of pre-trained.
Full size table

Human machine collaborative annotation framework

In order to balance labeling accuracy and expert annotation cost, we propose a framework for automatically generating Pseudo-expert label, which consists of two modules: deep learning module and manual correction medules. Its framework is shown in the following Fig. 5.

Fig. 5
figure 5

Detailed model of the HMCAF: OCTA-NET26, U-Net27, AttResU-Net28 and AttU-Net29.

Full size image

Machine learning

For the beginner learner, we select the basic framework model OCTA-NET26, U-Net27, AttResU-Net28 and AttU-Net29, which are pre-train on the ROSE-1 dataset using two-level labels of the SVC, to generate primary labels. However, the results are controversial due to the nature of the data for tiny blood vessel. Subsequently, we used the weighted fusion method to integrate the results of different models to obtain an improved fine vessel fusion label, as shown in Fig. 6.

Fig. 6
figure 6

Results of different models and the fusion labels.

Full size image

Human correction

Through deep learning, we have achieved labeling results with a certain level of accuracy. However, the inherent characteristics of diseases, such as patients’ poor fixation and other factors. Hence, to further enhance the accuracy and clinical applicability of our results, our ophthalmologists employ Labelme software (https://github.com/labelmeai/labelme)30 to correct the fusion label results (including additions, deletions, modifications, etc.). An example of artifacts and a comparison chart of expert corrections are shown in the Fig. 7.

Fig. 7
figure 7

Diagram of the Expert Correction Module.

Full size image

Data Records

Soul is available on Figshare31. All personal information that could be used to identify the patients was removed before preparation. The dataset details are presented in Table 3, encompassing the mean age of patients, gender distribution proportions, and image counts.

Table 3 Basic statistics of Soul dataset.
Full size table

The corresponding diagram of the data subset is shown in the figure below.The dataset comprises projection maps, four types of text labels, and pixel-level labels constructed through human machine collaboration. In Fig. 6 on the left, the subgraph represents three surgeries denoted as s1 (includes s1t1, s1t2), s2 (includes s2t1, s2t2), and s3 (includes s3t1, s3t2) respectively, with a time interval greater than or equal to one month (i.e., s2 - s1 >= 1 month; s3 - s2 >= 1 month). The term “s1t1” refers to the first follow-up after the initial operation, while subsequent terms indicate different surgeries and their respective follow-up visits. Herein, it is ensured that s1 - s1t1 >= 7 days and s1 - s1t2 >= 14 days.

In order to satisfy one operation and two follow-up visits, 36 individuals must be included in a total of 48; similarly, for the subgraph depicted in Fig. 8 on the left, Soul’s structure is illustrated. Additionally, Fig. 9 provides a sample set of data for pre-operative and post-operative follow-up diagnoses with multiple samples.

Fig. 8
figure 8

The population distribution of the correspondence dataset, along with its folder hierarchy.

Full size image
Fig. 9
figure 9

Pre-operative and post-operative images showcasing annotated illustrations.

Full size image

Technical Validation

Data

The OCTA scans were selected retrospectively by a search of the digital files from a public university hospital and a private imaging practice. For the Optovue Angio OCT device,the examinations were performed from Nov.20.2020 to Dec.30.2021.

Inclusion criteria

OCTA images was performed using the angio retina mode. A 6 × 6mm scan, centered on the fovea, was performed in all study eyes for diagnosing branch retinal vein occlusion. File availability in the Picture Archive and Communication System (PACS) from medical center.

Exclusion criteria

Artifacts that prevented the ophthalmologist from visually interpreting the images. Examinations with fully or partially corrupted files. There was no restriction on age, patient status(inpatients or outpatients) or any other inclusion or exclusion criteria different from those mentioned.

Model selection

We chose the widely recognized U-Net basic model and the classical attention module model as our baseline models, along with the latest retinal vascular segmentation model OCTA, to ensure that our reusable baseline models are not only classical but also incorporate state-of-the-art research features.

Docter announction

The label results of model reuse were manually reviewed and corrected by ophthalmologists specialized in BRVO-related diseases.

Usage Notes

Readers who wish to download the dataset may do so at figshare31. We hope that this published dataset will be available to more researchers and encourage more authors to publish their optimized codes and models, which will contribute to the development and advancement of the field of BRVO disease. Although the dataset we constructed does not have a significant improvement in the number of datasets, our dataset gives multiple follow-up data for the same patient, including image data and corresponding text labels. This improvement allows us to use multiple follow-up data to assist doctors in disease diagnosis or prognosis assessment, and to use image-text multimodal data for related research. The HMCAF we built can greatly reduce manual input while ensuring accuracy. The framework we built is not limited to a certain kind of data, it has a good generalization effect on various clinical data, it is not only suitable for OCTA image data, but also can be well generalized to CT data, MRI data and so on; This process can achieve better results in the relatively precise segmentation task of blood vessels, so the segmentation of lesions such as optic discs and pulmonary nodules can also achieve better results. Clear instructions for acquisition and segmentations are given for readers who wish to increase the size of the dataset.