Abstract
Purpose
To evaluate the inter-rater reliability for identification of complete retinal pigment epithelium and outer retinal atrophy (cRORA) on SD-OCT images as defined by the Classification of Atrophy Meetings (CAM) group.
Methods
Fifty images of anonymized SD-OCT line scans of eyes with cRORA due to AMD were selected. Each .tiff image was saved in both black-on-white (BW) and white-on-black (WB) format. Five retina-trained clinicians graded both sets of images twice for the diagnosis of cRORA based on the CAM group definition. Fleiss kappa statistic was calculated for inter-rater reliability and Cohen’s kappa statistic for intra-grader and inter-grader reliability between any two graders.
Results
The inter-grader reliability varied from as low as 0.28 to 0.92 for WB images and 0.34 to 0.86 for BW images. However, the inter-grader and intra-grader agreement was ĸ WB 0.92; ĸ BW 0.86 and ĸ 0.92 respectively, for graders accustomed to the CAM criteria. Fleiss kappa was ĸ 0.49 (p value < 0.0001) for WB images and ĸ 0.34 (p value < 0.0001 for BW images. Overall, the agreement was better using WB images for all parameters except RPE attenuation/loss.
Conclusion
There is significant variability in diagnosis of cRORA on SD-OCT by retina-trained ophthalmologists in the real world. The study highlights the need for training to recognise the different features of cRORA prior to its implementation in clinical practice.
Similar content being viewed by others
Log in or create a free account to read this content
Gain free access to this article, as well as selected content from this journal and more on nature.com
or
References
Schmitz-Valckenberg S, Sadda S, Staurenghi G, Chew EY, Fleckenstein M, Holz FG. Geographic atrophy: semantic considerations and literature review. Retina. 2016;36:2250–64.
Klein R, Davis MD, Magli YL, Segal P, Klein BE, Hubbard L. The Wisconsin age-related maculopathy grading system. Ophthalmology. 1991;98:1128–34.
Age-Related Eye Disease Study Research Group. The age-related eye disease study system for classifying age-related macular degeneration from stereoscopic colour fundus photographs: the age-related eye disease study report number 6. Am J Ophthalmol. 2001;132:668–81.
Gass JDM. Stereoscopic atlas of macular diseases. 1st ed. St. Louis: The C. V. Mosby Company; 1970.
Bird AC, Bressler NM, Bressler SB, Chisholm IH, Coscas G, Davis MD, et al. An international classification and grading system for age-related maculopathy and age-related macular degeneration. The International ARM Epidemiological Study Group. Surv Ophthalmol. 1995;39:367–74.
Fleckenstein M, Charbel Issa P, Helb HM, Schmitz-Valckenberg S, Finger RP, Scholl HP, et al. High-resolution spectral domain-OCT imaging in geographic atrophy associated with age-related macular degeneration. Investig Ophthalmol Vis Sci. 2008;49:4137–44.
Schmitz-Valckenberg S, Fleckenstein M, Göbel AP, Sehmi K, Fitzke FW, Holz FG, et al. Evaluation of autofluorescence imaging with the scanning laser ophthalmoscope and the fundus camera in age-related geographic atrophy. Am J Ophthalmol. 2008;146:183–92.
Shen LL, Liu F, Nardini HG, Del Priore LV. Reclassification of fundus autofluorescence patterns surrounding geographic atrophy based on progression rate: a systematic review and meta-analysis. Retina. 2019;39:1829–39.
Biarnés M, Forero CG, Arias L, Alonso J, Monés J. Reappraisal of geographic atrophy patterns seen on fundus autofluorescence using a latent class analysis approach. Investig Ophthalmol Vis Sci. 2014;55:8302–8.
Sadda SR, Guymer R, Holz FG, Schmitz-Valckenberg S, Curcio CA, Bird AC, et al. Consensus definition for atrophy associated with age-related macular degeneration on OCT: classification of atrophy report 3. Ophthalmology. 2018;125:537–48.
StatsToDo. Kappa (Cohen & Fleiss) for ordinal data program. 2020. https://www.statstodo.com/CohenFleissKappa.Pgm.php.
McHugh ML. Interrater reliability: the kappa statistic. Biochem Med. 2012;22:276–82.
Mendonça LSM, Perrott-Reynolds R, Schwartz R, Madi HA, Cronbach N, Gendelman I, et al. Deliberations of an International Panel of Experts on OCTA nomenclature of nAMD. Ophthalmology. 2020;S0161-6420:31198–2.
Acknowledgements
SC and RR are supported by the ORNATE India grant (GCRF UKRI MR/P207881/1). The research was supported by the NIHR Biomedical Research Centre and the Clinical Research Facility at Moorfields Eye Hospital NHS Foundation trust and UCL Institute of Technology.
Funding
The study is funded by Fight For Sight (Grant code 1905).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Chandra, S., Rasheed, R., Sen, P. et al. Inter-rater reliability for diagnosis of geographic atrophy using spectral domain OCT in age-related macular degeneration. Eye 36, 392–397 (2022). https://doi.org/10.1038/s41433-021-01490-5
Received:
Revised:
Accepted:
Published:
Issue date:
DOI: https://doi.org/10.1038/s41433-021-01490-5
This article is cited by
-
Clinical validation for automated geographic atrophy monitoring on OCT under complement inhibitory treatment
Scientific Reports (2023)
-
Measuring atrophy in age-related macular degeneration: why does reliability matter?
Eye (2022)
-
Validation of central serous chorioretinopathy multimodal imaging-based classification system
Graefe's Archive for Clinical and Experimental Ophthalmology (2022)
