Abstract
Objective
To introduce a new method of grading optic nerve stereo disc photographs and evaluate reproducibility of assessments by non-physician graders in a reading center.
Methods
Three non-physician graders, experienced in grading features of the retina but not the optic nerve head (ONH), were trained by glaucoma specialists to assess digital stereo color images of the ONH. These graders assessed a total of 2554 digital stereo disc images from glaucoma cases and controls participating in the Primary Open-Angle African American Glaucoma Genetics (POAAGG) study by outlining the optic cup and disc. Inter-grader reproducibility of area, height, and width measurements was analyzed.
Results
Among all images, the intraclass correlation (95% confidence interval) was 0.90 (0.89, 0.90) for the cup area using only color cues; 0.92 (0.91, 0.92) for the cup area using contour and vascular cues; and 0.99 (0.99, 0.99) for the optic disc area. The intraclass correlation for cup-to-disc ratio (CDR) was 0.61 (0.58, 0.63), as determined by the ratio of optic cup area to optic disc area (using contour and vascular cues). The CDR difference by graders for area was ≤ 0.1 in 65% of images using color/vascular cues and ≤0.1 in 71% of images using color cues.
Conclusions
After adequate training, non-physician graders were able to measure the optic nerve CDR with high inter-grader reliability.
Similar content being viewed by others
Log in or create a free account to read this content
Gain free access to this article, as well as selected content from this journal and more on nature.com
or
References
Gandhi M, Dubey S. Evaluation of the optic nerve head in glaucoma. J Curr Glaucoma Pract. 2013;7:106–14.
Tham YC, Li X, Wong TY, Quigley HA, Aung T, Cheng CY. Global prevalence of glaucoma and projections of glaucoma burden through 2040: a systematic review and meta-analysis. Ophthalmology. 2014;121:2081–90.
Spaeth GL, Lopes JF, Junk AK, Grigorian AP, Henderer J. Systems for staging the amount of optic nerve damage in glaucoma: a critical review and new material. Surv Ophthalmol. 2006;51:293–315.
Hitchings RA, Spaeth GL. The optic disc in glaucoma. I: Classification. Br J Ophthalmol. 1976;60:778–85.
Lichter PR. Variability of expert observers in evaluating the optic disc. Trans Am Ophthalmol Soc. 1976;74:532–72.
Varma R, Steinmann WC, Scott IU. Expert agreement in evaluating the optic disc for glaucoma. Ophthalmology. 1992;99:215–21.
Abrams LS, Scott IU, Spaeth GL, Quigley HA, Varma R. Agreement among optometrists, ophthalmologists, and residents in evaluating the optic disc for glaucoma. Ophthalmology. 1994;101:1662–7.
Shiose Y. Quantitative analysis of “optic cup” and its clinical application. III. A new diagnostic criterion for glaucoma using “quantitative disc pattern” (Shiose) (author’s transl). Nippon Ganka Gakkai Zasshi. 1975;79:445–61.
Read RM, Spaeth GL. The practical clinical appraisal of the optic disc in glaucoma: the natural history of cup progression and some specific disc-field correlations. Trans Am Acad Ophthalmol Otolaryngol. 1974;78:OP255–74.
Schwartz B. Cupping and pallor of the optic disc. Arch Ophthalmol. 1973;89:272–7.
Richardson KT. Glaucoma and glaucoma suspects. Glaucoma: Conceptions of a disease, pathogenesis, diagnosis, therapy. 1978; 2–6.
Nesterov AP, Listopadova NA. Classification of physiological and glaucomatous extraction of the optic disk. Vestn Oftalmol. 1981;2:17–22.
Jonas JB, Gusek GC, Naumann GO. Optic disc morphometry in chronic primary open-angle glaucoma. II. Correlation of the intrapapillary morphometric data to visual field indices. Graefes Arch Clin Exp Ophthalmol. 1988;226:531–8.
Spaeth GL, Henderer J, Liu C, Kesen M, Altangerel U, Bayer A, et al. The disc damage likelihood scale: reproducibility of a new method of estimating the amount of optic nerve damage caused by glaucoma. Trans Am Ophthalmol Soc. 2002;100:181–5.
Jampel HD, Friedman D, Quigley H, Vitale S, Miller R, Knezevich F, et al. Agreement among glaucoma specialists in assessing progressive disc changes from photographs in open-angle glaucoma patients. Am J Ophthalmol. 2009;147:39–44.e1.
Montgomery DM, Craig JP. Optic disc interpretation in glaucoma: is confidence misplaced? Ophthalmic Physiol Opt. 1993;13:383–6.
Harper R, Reeves B, Smith G. Observer variability in optic disc assessment: implications for glaucoma shared care. Ophthalmic Physiol Opt. 2000;20:265–73.
Gaasterland DE, Blackwell B, Dally LG, Caprioli J, Katz LJ, Ederer F, et al. The Advanced Glaucoma Intervention Study (AGIS): 10. Variability among academic glaucoma subspecialists in assessing optic disc notching. Trans Am Ophthalmol Soc. 2001;99:177–84.
Vermeer KA, Vos FM, Lemij HG, Vossepoel AM. Detecting glaucomatous wedge shaped defects in polarimetric images. Med Image Anal. 2003;7:503–11.
Correnti AJ, Wollstein G, Price LL, Schuman JS. Comparison of optic nerve head assessment with a digital stereoscopic camera (discam), scanning laser ophthalmoscopy, and stereophotography. Ophthalmology. 2003;110:1499–505.
Schuman JS, Wollstein G, Farra T, Hertzmark E, Aydin A, Fujimoto JG, et al. Comparison of optic nerve head measurements obtained by optical coherence tomography and confocal scanning laser ophthalmoscopy. Am J Ophthalmol. 2003;135:504–12.
Zangwill LM, Weinreb RN, Beiser JA, Berry CC, Cioffi GA, Coleman AL, et al. Baseline topographic optic disc measurements are associated with the development of primary open-angle glaucoma: the Confocal Scanning Laser Ophthalmoscopy Ancillary Study to the Ocular Hypertension Treatment Study. Arch Ophthalmol. 2005;123:1188–97.
Banister K, Boachie C, Bourne R, Cook J, Burr JM, Ramsay C, et al. Can automated imaging for optic disc and retinal nerve fiber layer analysis aid glaucoma detection? Ophthalmology. 2016;123:930–8.
Reus NJ, Lemij HG, Garway-Heath DF, Airaksinen PJ, Anton A, Bron AM, et al. Clinical assessment of stereoscopic optic disc photographs for glaucoma: the European Optic Disc Assessment Trial. Ophthalmology. 2010;117:717–23.
Charlson ES, Sankar PS, Miller-Ellis E, Regina M, Fertig R, Salinas J, et al. The primary open-angle african american glaucoma genetics study: baseline demographics. Ophthalmology. 2015;122:711–20.
CATT Research Group, Martin DF, Maguire MG, Ying GS, Grunwald JE, Fine SL, et al. Ranibizumab and bevacizumab for neovascular age-related macular degeneration. N Engl J Med. 2011;364:1897–908.
Feuer WJ, Parrish RK 2nd, Schiffman JC, Anderson DR, Budenz DL, Wells MC, et al. The Ocular Hypertension Treatment Study: reproducibility of cup/disk ratio measurements over time at an optic disc reading center. Am J Ophthalmol. 2002;133:19–28.
Azuara-Blanco A, Katz LJ, Spaeth GL, Vernon SA, Spencer F, Lanzl IM. Clinical agreement among glaucoma experts in the detection of glaucomatous changes of the optic disk using simultaneous stereoscopic photographs. Am J Ophthalmol. 2003;136:949–50.
Breusegem C, Fieuws S, Stalmans I, Zeyen T. Agreement and accuracy of non-expert ophthalmologists in assessing glaucomatous changes in serial stereo optic disc photographs. Ophthalmology. 2011;118:742–6.
Parrish RK 2nd, Schiffman JC, Feuer WJ, Anderson DR, Budenz DL, Wells-Albornoz MC, et al. Test-retest reproducibility of optic disk deterioration detected from stereophotographs by masked graders. Am J Ophthalmol. 2005;140:762–4.
Funding
This work was supported by the National Eye Institute, Bethesda, Maryland (grant #1RO1EY023557-01) and the Department of Ophthalmology at the Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA. Funds also come from the Vision Research Core Grant (P30 EY001583), F.M. Kirby Foundation, Research to Prevent Blindness, The UPenn Hospital Board of Women Visitors, The Paul and Evanina Bell Mackall Foundation Trust, and the National Eye Institute, National Institutes of Health, Department of Health and Human Services, under eyeGENETM and contract Nos. HHSN260220700001C and HHSN263201200001C. The sponsor or funding organization had no role in the design or conduct of this research.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Addis, V., Oyeniran, E., Daniel, E. et al. Non-physician grader reliability in measuring morphological features of the optic nerve head in stereo digital images. Eye 33, 838–844 (2019). https://doi.org/10.1038/s41433-018-0332-8
Received:
Revised:
Accepted:
Published:
Issue date:
DOI: https://doi.org/10.1038/s41433-018-0332-8
This article is cited by
-
Remote screening of retinal and optic disc diseases using handheld nonmydriatic cameras in programmed routine occupational health checkups onsite at work centers
Graefe's Archive for Clinical and Experimental Ophthalmology (2021)
