Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Review Article
  • Published:

Machine intelligence in non-invasive endocrine cancer diagnostics

Nature Reviews Endocrinology volume 18, pages 81–95 (2022)Cite this article

Subjects

Abstract

Artificial intelligence (AI) has illuminated a clear path towards an evolving health-care system replete with enhanced precision and computing capabilities. Medical imaging analysis can be strengthened by machine learning as the multidimensional data generated by imaging naturally lends itself to hierarchical classification. In this Review, we describe the role of machine intelligence in image-based endocrine cancer diagnostics. We first provide a brief overview of AI and consider its intuitive incorporation into the clinical workflow. We then discuss how AI can be applied for the characterization of adrenal, pancreatic, pituitary and thyroid masses in order to support clinicians in their diagnostic interpretations. This Review also puts forth a number of key evaluation criteria for machine learning in medicine that physicians can use in their appraisals of these algorithms. We identify mitigation strategies to address ongoing challenges around data availability and model interpretability in the context of endocrine cancer diagnosis. Finally, we delve into frontiers in systems integration for AI, discussing automated pipelines and evolving computing platforms that leverage distributed, decentralized and quantum techniques.

Key points

  • Developments in machine intelligence have been made possible by the increase in data ubiquity and computing power and have the potential to enhance image segmentation, analysis and workflow in non-invasive endocrine cancer diagnostics.

  • Improved adherence to consensus reporting standards and evaluation criteria in artificial intelligence (AI) for medical image analysis is urgently needed in the field of endocrine cancer diagnostics as this will enable meaningful cross-study comparison.

  • A centralized inventory to track diagnostic algorithms in oncologic endocrinology that are in active clinical use would improve performance auditing and algorithm stewardship.

  • The looming risk of excessive intervention in endocrine cancers can be addressed with the improved detection facilitated by AI, possibly via correlation with prognostic data for improved risk stratification.

  • Poor data availability continues to stymie the development of robust machine learning applications, particularly in rare endocrine cancers; solutions to this problem might include database curation, pre-training techniques and workflow automation.

  • Other breakthroughs in machine intelligence will come with the exploration of alternative computing frameworks, such as decentralized, distributed and quantum networks, that might enhance model training and efficiency.

Access through your institution
Buy or subscribe

This is a preview of subscription content, access via your institution

Access options

Access through your institution

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Integrative diagnostics.
Fig. 2: Computer vision workflow.
Fig. 3: A convolutional neural network.
Fig. 4: Real-time analytics with automatic picture archiving and communications systems integration.
Fig. 5: Exploring alternative computing platforms.

Similar content being viewed by others

References

  1. Goodfellow, I., Bengio, Y., Courville, A. & Bengio, Y. Deep learning. Vol. 1 (MIT Press, 2016).

  2. Esteva, A. et al. A guide to deep learning in healthcare. Nat. Med. 25, 24–29 (2019). This Review provides an excellent primer on deep learning applications in medicine that covers a variety of modalities, including clinical, imaging, text and mixed data.

    CAS  PubMed  Google Scholar 

  3. Hosny, A., Parmar, C., Quackenbush, J., Schwartz, L. H. & Aerts, H. J. W. L. Artificial intelligence in radiology. Nat. Rev. Cancer 18, 500–510 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  4. Luo, Y. et al. Preoperative prediction of pancreatic neuroendocrine neoplasms grading based on enhanced computed tomography imaging: validation of deep learning with a convolutional neural network. Neuroendocrinology 110, 338–350 (2020). This paper finds the deep learning convolutional neural network approach to achieve the highest area under the curve in differentiating pancreatic NET grade 1–2 from grade 3 tumours, although convolutional neural network performance was not statistically different from that of the traditional machine learning models included in the study.

    CAS  PubMed  Google Scholar 

  5. Qian, Y. et al. A novel diagnostic method for pituitary adenoma based on magnetic resonance imaging using a convolutional neural network. Pituitary 23, 246–252 (2020). A deep learning technique using convolutional neural networks to differentiate patients with pituitary adenoma from a mixed control group with both healthy and sellar lesion MRI scans.

    CAS  PubMed  Google Scholar 

  6. Li, X. et al. Diagnosis of thyroid cancer using deep convolutional neural network models applied to sonographic images: a retrospective, multicohort, diagnostic study. Lancet Oncol. 20, 193–201 (2019). A large cohort study using a convolutional neural network-based approach to thyroid nodule diagnosis on ultrasound demonstrating comparable sensitivity and improved specificity when compared with a group of expert radiologists.

    PubMed  Google Scholar 

  7. Wang, L. et al. Automatic thyroid nodule recognition and diagnosis in ultrasound imaging with the YOLOv2 neural network. World J. Surg. Oncol. 17, 12 (2019).

    PubMed  PubMed Central  Google Scholar 

  8. Gerlinger, M. et al. Intratumor heterogeneity and branched evolution revealed by multiregion sequencing. N. Engl. J. Med. 366, 883–892 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  9. Chmielik, E. et al. Heterogeneity of thyroid cancer. Pathobiology 85, 117–129 (2018).

    PubMed  Google Scholar 

  10. Topol, E. J. Individualized medicine from prewomb to tomb. Cell 157, 241–253 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  11. Obermeyer, Z. & Emanuel, E. J. Predicting the future - big data, machine learning, and clinical medicine. N. Engl. J. Med. 375, 1216–1219 (2016).

    PubMed  PubMed Central  Google Scholar 

  12. Rao, A. et al. A combinatorial radiographic phenotype may stratify patient survival and be associated with invasion and proliferation characteristics in glioblastoma. J. Neurosurg. 124, 1008–1017 (2016).

    CAS  PubMed  Google Scholar 

  13. Yamamoto, S., Maki, D. D., Korn, R. L. & Kuo, M. D. Radiogenomic analysis of breast cancer using MRI: a preliminary study to define the landscape. AJR Am. J. Roentgenol. 199, 654–663 (2012).

    PubMed  Google Scholar 

  14. Aerts, H. J. et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat. Commun. 5, 4006 (2014).

    CAS  PubMed  Google Scholar 

  15. Zhao, C. K. et al. A comparative analysis of two machine learning-based diagnostic patterns with thyroid imaging reporting and data system for thyroid nodules: diagnostic performance and unnecessary biopsy rate. Thyroid 31, 470–481 (2021).

    CAS  PubMed  Google Scholar 

  16. Gulshan, V. et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316, 2402–2410 (2016).

    PubMed  Google Scholar 

  17. Zhou, H. et al. Machine learning reveals multimodal MRI patterns predictive of isocitrate dehydrogenase and 1p/19q status in diffuse low- and high-grade gliomas. J. Neurooncol. 142, 299–307 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  18. Liang, W. et al. Development and validation of a clinical risk score to predict the occurrence of critical illness in hospitalized patients with COVID-19. JAMA Intern. Med. 180, 1081–1089 (2020).

    CAS  PubMed  Google Scholar 

  19. Cohen, J. D. et al. Detection and localization of surgically resectable cancers with a multi-analyte blood test. Science 359, 926–930 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  20. Davis, R. J. et al. Pan-cancer transcriptional signatures predictive of oncogenic mutations reveal that Fbw7 regulates cancer cell oxidative metabolism. Proc. Natl Acad. Sci. USA 115, 5462–5467 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  21. Chang, E. K. et al. Defining a patient population with cirrhosis: an automated algorithm with natural language processing. J. Clin. Gastroenterol. 50, 889–894 (2016).

    PubMed  Google Scholar 

  22. Bedi, G. et al. Automated analysis of free speech predicts psychosis onset in high-risk youths. NPJ Schizophrenia 1, 15030 (2015).

    PubMed  PubMed Central  Google Scholar 

  23. Yu, P. et al. FGF-dependent metabolic control of vascular development. Nature 545, 224–228 (2017).

    CAS  PubMed  PubMed Central  Google Scholar 

  24. Samuel, A. L. Some studies in machine learning using the game of checkers. IBM J. Res. Dev. 3, 210–229 (1959).

    Google Scholar 

  25. Kumar, V. et al. Radiomics: the process and the challenges. Magn. Reson. Imaging 30, 1234–1248 (2012).

    PubMed  PubMed Central  Google Scholar 

  26. Erickson, B. J., Korfiatis, P., Akkus, Z. & Kline, T. L. Machine learning for medical imaging. Radiographics 37, 505–515 (2017).

    PubMed  Google Scholar 

  27. Guo, Y., Gao, Y. & Shen, D. Deformable MR prostate segmentation via deep feature learning and sparse patch matching. IEEE Trans. Med. Imaging 35, 1077–1089 (2016).

    PubMed  Google Scholar 

  28. Wu, J. et al. A deep Boltzmann machine-driven level set method for heart motion tracking using cine MRI images. Med. Image Anal. 47, 68–80 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  29. Sutton, R. S. & Barto, A. G. Introduction to Reinforcement Learning. Vol. 135 (MIT Press, 1998).

  30. Bengio, Y., Courville, A. & Vincent, P. Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35, 1798–1828 (2013).

    PubMed  Google Scholar 

  31. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

    CAS  PubMed  Google Scholar 

  32. Litjens, G. et al. A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017).

    PubMed  Google Scholar 

  33. Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017).

    CAS  PubMed  PubMed Central  Google Scholar 

  34. Naceur, M. B., Saouli, R., Akil, M. & Kachouri, R. Fully automatic brain tumor segmentation using end-to-end incremental deep neural networks in MRI images. Comput. Methods Prog. Biomed. 166, 39–49 (2018).

    Google Scholar 

  35. Lee, M. J. et al. Benign and malignant adrenal masses: CT distinction with attenuation coefficients, size, and observer analysis. Radiology 179, 415–418 (1991).

    CAS  PubMed  Google Scholar 

  36. Song, J. H., Chaudhry, F. S. & Mayo-Smith, W. W. The incidental adrenal mass on CT: prevalence of adrenal disease in 1,049 consecutive adrenal masses in patients with no known malignancy. AJR Am. J. Roentgenol. 190, 1163–1168 (2008).

    PubMed  Google Scholar 

  37. Zeiger, M. et al. American Association of Clinical Endocrinologists and American Association of Endocrine Surgeons medical guidelines for the management of adrenal incidentalomas. Endocr. Pract. 15, 1–20 (2009).

    PubMed  Google Scholar 

  38. Bae, K. T., Fuangtharnthip, P., Prasad, S. R., Joe, B. N. & Heiken, J. P. Adrenal masses: CT characterization with histogram analysis method. Radiology 228, 735–742 (2003).

    PubMed  Google Scholar 

  39. Ho, L. M., Paulson, E. K., Brady, M. J., Wong, T. Z. & Schindera, S. T. Lipid-poor adenomas on unenhanced CT: does histogram analysis increase sensitivity compared with a mean attenuation threshold? Am. J. Roentgenol. 191, 234–238 (2008).

    Google Scholar 

  40. Umanodan, T. et al. ADC histogram analysis for adrenal tumor histogram analysis of apparent diffusion coefficient in differentiating adrenal adenoma from pheochromocytoma. J. Magn. Reson. Imaging 45, 1195–1203 (2017).

    PubMed  Google Scholar 

  41. Tüdös, Z. & Čtvrtlík, F. Possible impact of CT histogram analysis in incidentally discovered adrenal masses. Abdom. Radiol. 45, 2937–2938 (2020).

    Google Scholar 

  42. Alobaidli, S. et al. The role of texture analysis in imaging as an outcome predictor and potential tool in radiotherapy treatment planning. Br. J. Radiol. 87, 20140369 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  43. Parekh, V. S. & Jacobs, M. A. Deep learning and radiomics in precision medicine. Expert Rev. Precis. Med. Drug Dev. 4, 59–72 (2019).

    PubMed  PubMed Central  Google Scholar 

  44. Ganeshan, B. & Miles, K. A. Quantifying tumour heterogeneity with CT. Cancer Imaging 13, 140–149 (2013).

    PubMed  PubMed Central  Google Scholar 

  45. Nieman, L. K. Approach to the patient with an adrenal incidentaloma. J. Clin. Endocrinol. Metab. 95, 4106–4113 (2010).

    CAS  PubMed  PubMed Central  Google Scholar 

  46. Iñiguez-Ariza, N. M. et al. Clinical, biochemical, and radiological characteristics of a single-center retrospective cohort of 705 large adrenal tumors. Mayo Clin. Proc. Innov. Qual. Outcomes 2, 30–39 (2018).

    PubMed  Google Scholar 

  47. Angeli, A., Osella, G., Alì, A. & Terzolo, M. Adrenal incidentaloma: an overview of clinical and epidemiological data from the National Italian Study Group. Horm. Res. 47, 279–283 (1997).

    CAS  PubMed  Google Scholar 

  48. Elmohr, M. M. et al. Machine learning-based texture analysis for differentiation of large adrenal cortical tumours on CT. Clin. Radiol. 74, 818.e1–818.e7 (2019). This study establishes a radiomics signature to differentiate large adrenal tumours using random forest-based machine learning feature extraction coupled with CT attenuation score; model performance exceeded that of two expert radiologists.

    CAS  Google Scholar 

  49. Korobkin, M. et al. Differentiation of adrenal adenomas from nonadenomas using CT attenuation values. AJR Am. J. Roentgenol. 166, 531–536 (1996).

    CAS  PubMed  Google Scholar 

  50. Patel, J., Davenport, M. S., Cohan, R. H. & Caoili, E. M. Can established CT attenuation and washout criteria for adrenal adenoma accurately exclude pheochromocytoma? Am. J. Roentgenol. 201, 122–127 (2013).

    Google Scholar 

  51. Northcutt, B. G., Trakhtenbroit, M. A., Gomez, E. N., Fishman, E. K. & Johnson, P. T. Adrenal adenoma and pheochromocytoma: comparison of multidetector CT venous enhancement levels and washout characteristics. J. Comput. Assist. Tomogr. 40, 194–200 (2016).

    PubMed  Google Scholar 

  52. Yi, X. et al. Adrenal incidentaloma: machine learning-based quantitative texture analysis of unenhanced CT can effectively differentiate sPHEO from lipid-poor adrenal adenoma. J. Cancer 9, 3577–3582 (2018).

    PubMed  PubMed Central  Google Scholar 

  53. Yi, X. et al. Radiomics improves efficiency for differentiating subclinical pheochromocytoma from lipid-poor adenoma: a predictive, preventive and personalized medical approach in adrenal incidentalomas. EPMA J. 9, 421–429 (2018). This study uses machine learning with a LASSO model to differentiate subclinical pheochromocytoma from lipid-poor adenomas on CT with a sensitivity of 90% and sensitivity of 99%, albeit without an expert radiologist comparison group.

    PubMed  PubMed Central  Google Scholar 

  54. Romeo, V. et al. Characterization of adrenal lesions on unenhanced MRI using texture analysis: a machine-learning approach. J. Magn. Reson. Imaging 48, 198–204 (2018).

    PubMed  Google Scholar 

  55. Barstugan, M., Ceylan, R., Asoglu, S., Cebeci, H. & Koplay, M. Adrenal tumor characterization on magnetic resonance images. Int. J. Imaging Syst. Technol. 30, 252–265 (2020).

    Google Scholar 

  56. Koyuncu, H., Ceylan, R., Asoglu, S., Cebeci, H. & Koplay, M. An extensive study for binary characterisation of adrenal tumours. Med. Biol. Eng. Comput. 57, 849–862 (2019).

    PubMed  Google Scholar 

  57. Henley, D. J., van Heerden, J. A., Grant, C. S., Carney, J. A. & Carpenter, P. C. Adrenal cortical carcinoma — a continuing challenge. Surgery 94, 926–931 (1983).

    CAS  PubMed  Google Scholar 

  58. Dasari, A. et al. Trends in the incidence, prevalence, and survival outcomes in patients with neuroendocrine tumors in the United States. JAMA Oncol. 3, 1335–1342 (2017).

    PubMed  PubMed Central  Google Scholar 

  59. Halfdanarson, T. R., Rabe, K. G., Rubin, J. & Petersen, G. M. Pancreatic neuroendocrine tumors (PNETs): incidence, prognosis and recent trend toward improved survival. Ann. Oncol. 19, 1727–1733 (2008).

    CAS  PubMed  PubMed Central  Google Scholar 

  60. Genç, C. G. et al. A new scoring system to predict recurrent disease in grade 1 and 2 nonfunctional pancreatic neuroendocrine tumors. Ann. Surg. 267, 1148–1154 (2018).

    PubMed  Google Scholar 

  61. Modlin, I. M. et al. Gastroenteropancreatic neuroendocrine tumours. Lancet Oncol. 9, 61–72 (2008).

    CAS  PubMed  Google Scholar 

  62. Zerbi, A. et al. Clinicopathological features of pancreatic endocrine tumors: a prospective multicenter study in Italy of 297 sporadic cases. Am. J. Gastroenterol. 105, 1421–1429 (2010).

    PubMed  Google Scholar 

  63. Manfredi, R. et al. Non-hyperfunctioning neuroendocrine tumours of the pancreas: MR imaging appearance and correlation with their biological behaviour. Eur. Radiol. 23, 3029–3039 (2013).

    PubMed  Google Scholar 

  64. Inzani, F., Petrone, G. & Rindi, G. The New World Health Organization Classification for Pancreatic Neuroendocrine Neoplasia. Endocrinol. Metab. Clin. North. Am. 47, 463–470 (2018).

    PubMed  Google Scholar 

  65. Rindi, G. & Wiedenmann, B. Neuroendocrine neoplasms of the gut and pancreas: new insights. Nat. Rev. Endocrinol. 8, 54 (2012).

    Google Scholar 

  66. Oronsky, B., Ma, P. C., Morgensztern, D. & Carter, C. A. Nothing but NET: a review of neuroendocrine tumors and carcinomas. Neoplasia 19, 991–1002 (2017).

    CAS  PubMed  PubMed Central  Google Scholar 

  67. Lee, N. J., Hruban, R. H. & Fishman, E. K. Pancreatic neuroendocrine tumor: review of heterogeneous spectrum of CT appearance. Abdom. Radiol. 43, 3025–3034 (2018).

    Google Scholar 

  68. Karmazanovsky, G. et al. Nonhypervascular pancreatic neuroendocrine tumors: Spectrum of MDCT imaging findings and differentiation from pancreatic ductal adenocarcinoma. Eur. J. Radiol. 110, 66–73 (2019).

    PubMed  Google Scholar 

  69. Rösch, T. et al. Localization of pancreatic endocrine tumors by endoscopic ultrasonography. N. Engl. J. Med. 326, 1721–1726 (1992).

    PubMed  Google Scholar 

  70. Song, Y. et al. Multiple machine learnings revealed similar predictive accuracy for prognosis of PNETs from the surveillance, epidemiology, and end result database. J. Cancer 9, 3971–3978 (2018).

    PubMed  PubMed Central  Google Scholar 

  71. Saleh, M. et al. New frontiers in imaging including radiomics updates for pancreatic neuroendocrine neoplasms. Abdom. Radiol. https://doi.org/10.1007/s00261-020-02833-8 (2020).

    Article  Google Scholar 

  72. Zhao, Z. et al. CT-radiomic approach to predict G1/2 nonfunctional pancreatic neuroendocrine tumor. Acad. Radiol. 27, e272–e281 (2020).

    PubMed  Google Scholar 

  73. Liang, W. et al. A combined nomogram model to preoperatively predict histologic grade in pancreatic neuroendocrine tumors. Clin. Cancer Res. 25, 584–594 (2019).

    PubMed  Google Scholar 

  74. Gu, D. et al. CT radiomics may predict the grade of pancreatic neuroendocrine tumors: a multicenter study. Eur. Radiol. 29, 6880–6890 (2019).

    PubMed  Google Scholar 

  75. Gao, X. & Wang, X. Deep learning for World Health Organization grades of pancreatic neuroendocrine tumors on contrast-enhanced magnetic resonance images: a preliminary study. Int. J. Comput. Assist. Radiol. Surg. 14, 1981–1991 (2019).

    PubMed  Google Scholar 

  76. Choi, T. W., Kim, J. H., Yu, M. H., Park, S. J. & Han, J. K. Pancreatic neuroendocrine tumor: prediction of the tumor grade using CT findings and computerized texture analysis. Acta Radiol. 59, 383–392 (2018).

    CAS  PubMed  Google Scholar 

  77. Duan, H., Baratto, L. & Iagaru, A. The role of PET/CT in the imaging of pancreatic neoplasms. Semin. Ultrasound CT MR 40, 500–508 (2019).

    PubMed  Google Scholar 

  78. Zaharchuk, G. Next generation research applications for hybrid PET/MR and PET/CT imaging using deep learning. Eur. J. Nucl. Med. Mol. Imaging 46, 2700–2707 (2019).

    PubMed  PubMed Central  Google Scholar 

  79. Wei, L., Osman, S., Hatt, M. & El Naqa, I. Machine learning for radiomics-based multimodality and multiparametric modeling. Q. J. Nucl. Med. Mol. Imaging 63, 323–338 (2019).

    PubMed  Google Scholar 

  80. Hidalgo, M. Pancreatic cancer. N. Engl. J. Med. 362, 1605–1617 (2010).

    CAS  PubMed  Google Scholar 

  81. Cameron, J. L. et al. Factors influencing survival after pancreaticoduodenectomy for pancreatic cancer. Am. J. Surg. 161, 120–124 (1991).

    CAS  PubMed  Google Scholar 

  82. Guo, C. et al. The differentiation of pancreatic neuroendocrine carcinoma from pancreatic ductal adenocarcinoma: the values of CT imaging features and texture analysis. Cancer Imaging 18, 37 (2018).

    PubMed  PubMed Central  Google Scholar 

  83. Li, J. et al. Differentiation of atypical pancreatic neuroendocrine tumors from pancreatic ductal adenocarcinomas: Using whole-tumor CT texture analysis as quantitative biomarkers. Cancer Med. 7, 4924–4931 (2018).

    PubMed  PubMed Central  Google Scholar 

  84. Fu, M. et al. Hierarchical combinatorial deep learning architecture for pancreas segmentation of medical computed tomography cancer images. BMC Syst. Biol. 12, 56 (2018).

    PubMed  PubMed Central  Google Scholar 

  85. Man, Y., Huang, Y., Feng, J., Li, X. & Wu, F. Deep Q learning driven CT pancreas segmentation with geometry-aware U-net. IEEE Trans. Med. Imaging 38, 1971–1980 (2019).

    PubMed  Google Scholar 

  86. Heinrich, M. P., Blendowski, M. & Oktay, O. TernaryNet: faster deep model inference without GPUs for medical 3D segmentation using sparse and binary convolutions. Int. J. Comput. Assist. Radiol. Surg. 13, 1311–1320 (2018).

    PubMed  Google Scholar 

  87. Gibson, E. et al. Automatic multi-organ segmentation on abdominal CT with dense V-networks. IEEE Trans. Med. Imaging 37, 1822–1834 (2018).

    PubMed  PubMed Central  Google Scholar 

  88. Liang, Y. et al. Auto-segmentation of pancreatic tumor in multi-parametric MRI using deep convolutional neural networks. Radiother. Oncol. 145, 193–200 (2020).

    PubMed  Google Scholar 

  89. Corral, J. E. et al. Deep learning to classify intraductal papillary mucinous neoplasms using magnetic resonance imaging. Pancreas 48, 805–810 (2019).

    PubMed  Google Scholar 

  90. Kuwahara, T. et al. Usefulness of deep learning analysis for the diagnosis of malignancy in intraductal papillary mucinous neoplasms of the pancreas. Clin. Transl. Gastroenterol. 10, 1–8 (2019).

    CAS  PubMed  Google Scholar 

  91. Hussein, S., Kandel, P., Bolan, C. W., Wallace, M. B. & Bagci, U. Lung and pancreatic tumor characterization in the deep learning era: novel supervised and unsupervised learning approaches. IEEE Trans. Med. Imaging 38, 1777–1787 (2019).

    PubMed  Google Scholar 

  92. Molitch, M. E. Diagnosis and treatment of pituitary adenomas: a review. JAMA 317, 516–524 (2017).

    PubMed  Google Scholar 

  93. Melmed, S. Pituitary-tumor endocrinopathies. N. Engl. J. Med. 382, 937–950 (2020).

    CAS  PubMed  Google Scholar 

  94. Chahal, J. & Schlechte, J. Hyperprolactinemia. Pituitary 11, 141–146 (2008).

    CAS  PubMed  Google Scholar 

  95. Vilar, L., Vilar, C. F., Lyra, R., Lyra, R. & Naves, L. A. Acromegaly: clinical features at diagnosis. Pituitary 20, 22–32 (2017).

    PubMed  Google Scholar 

  96. Ntali, G. & Wass, J. A. Epidemiology, clinical presentation and diagnosis of non-functioning pituitary adenomas. Pituitary 21, 111–118 (2018).

    PubMed  Google Scholar 

  97. Amlashi, F. G. & Tritos, N. A. Thyrotropin-secreting pituitary adenomas: epidemiology, diagnosis, and management. Endocrine 52, 427–440 (2016).

    CAS  PubMed  Google Scholar 

  98. Varlamov, E. V., McCartney, S. & Fleseriu, M. Functioning pituitary adenomas — current treatment options and emerging medical therapies. Eur. Endocrinol. 15, 30–40 (2019).

    PubMed  PubMed Central  Google Scholar 

  99. Zamora, C. & Castillo, M. Sellar and parasellar imaging. Neurosurgery 80, 17–38 (2017).

    PubMed  Google Scholar 

  100. Connor, S. E. & Penney, C. C. MRI in the differential diagnosis of a sellar mass. Clin. Radiol. 58, 20–31 (2003).

    CAS  PubMed  Google Scholar 

  101. Kitajima, M. et al. Differentiation of common large sellar-suprasellar masses effect of artificial neural network on radiologists’ diagnosis performance. Acad. Radiol. 16, 313–320 (2009).

    PubMed  Google Scholar 

  102. Zhang, S. et al. Non-invasive radiomics approach potentially predicts non-functioning pituitary adenomas subtypes before surgery. Eur. Radiol. 28, 3692–3701 (2018).

    PubMed  Google Scholar 

  103. Zeynalova, A. et al. Preoperative evaluation of tumour consistency in pituitary macroadenomas: a machine learning-based histogram analysis on conventional T2-weighted MRI. Neuroradiology 61, 767–774 (2019).

    PubMed  Google Scholar 

  104. Fan, Y. et al. Preoperative noninvasive radiomics approach predicts tumor consistency in patients with acromegaly: development and multicenter prospective validation. Front. Endocrinol. 10, 403 (2019). This prospective, multi-institutional machine learning study evaluates pituitary adenoma consistency in patients with acromegaly using a support vector machine-derived radiomics signature found to have a higher diagnostic accuracy than clinical characteristics alone.

    Google Scholar 

  105. Zhu, H., Fang, Q., Huang, Y. & Xu, K. Semi-supervised method for image texture classification of pituitary tumors via CycleGAN and optimized feature extraction. BMC Med. Inf. Decis. Mak. 20, 215 (2020). This study uses multiple deep learning techniques for pituitary texture analysis including a generative adversarial network for data augmentation followed by unsupervised feature extraction with a convolutional neural network-based auto-encoder framework that is then fed into a convolutional recurrent neural network for classification.

    Google Scholar 

  106. Yamamoto, J. et al. Tumor consistency of pituitary macroadenomas: predictive analysis on the basis of imaging features with contrast-enhanced 3D FIESTA at 3T. Am. J. Neuroradiol. 35, 297–303 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  107. Iuchi, T., Saeki, N., Tanaka, M., Sunami, K. & Yamaura, A. MRI prediction of fibrous pituitary adenomas. Acta Neurochir. 140, 779–786 (1998).

    CAS  PubMed  Google Scholar 

  108. Niu, J. et al. Preoperative prediction of cavernous sinus invasion by pituitary adenomas using a radiomics method based on magnetic resonance images. Eur. Radiol. 29, 1625–1634 (2019).

    PubMed  Google Scholar 

  109. Staartjes, V. E. et al. Neural network-based identification of patients at high risk for intraoperative cerebrospinal fluid leaks in endoscopic pituitary surgery. J. Neurosurg. 133, 329–335 (2019).

    Google Scholar 

  110. Kitahara, C. M. & Sosa, J. A. The changing incidence of thyroid cancer. Nat. Rev. Endocrinol. 12, 646–653 (2016).

    PubMed  Google Scholar 

  111. Cabanillas, M. E., McFadden, D. G. & Durante, C. Thyroid cancer. Lancet 388, 2783–2795 (2016).

    CAS  PubMed  Google Scholar 

  112. Song, W. et al. Multitask cascade convolution neural networks for automatic thyroid nodule detection and recognition. IEEE J. Biomed. Health Inf. 23, 1215–1224 (2019).

    Google Scholar 

  113. Li, H. et al. An improved deep learning approach for detection of thyroid papillary cancer in ultrasound images. Sci. Rep. 8, 6600 (2018).

    PubMed  PubMed Central  Google Scholar 

  114. Ma, J., Wu, F., Zhu, J., Xu, D. & Kong, D. A pre-trained convolutional neural network based method for thyroid nodule diagnosis. Ultrasonics 73, 221–230 (2017).

    PubMed  Google Scholar 

  115. Lim, K. J. et al. Computer-aided diagnosis for the differentiation of malignant from benign thyroid nodules on ultrasonography. Acad. Radiol. 15, 853–858 (2008).

    PubMed  Google Scholar 

  116. Chi, J. et al. Thyroid nodule classification in ultrasound images by fine-tuning deep convolutional neural network. J. Digit. Imaging 30, 477–486 (2017).

    PubMed  PubMed Central  Google Scholar 

  117. Acharya, U. R. et al. Non-invasive automated 3D thyroid lesion classification in ultrasound: a class of ThyroScan™ systems. Ultrasonics 52, 508–520 (2012).

    PubMed  Google Scholar 

  118. Tessler, F. N. et al. ACR thyroid imaging, reporting and data system (TI-RADS): white paper of the ACR TI-RADS Committee. J. Am. Coll. Radiol. 14, 587–595 (2017).

    PubMed  Google Scholar 

  119. Zhang, B. et al. Machine learning-assisted system for thyroid nodule diagnosis. Thyroid 29, 858–867 (2019).

    PubMed  Google Scholar 

  120. Zhu, L. C. et al. A model to discriminate malignant from benign thyroid nodules using artificial neural network. PLoS One 8, e82211 (2013).

    PubMed  PubMed Central  Google Scholar 

  121. Song, G., Xue, F. & Zhang, C. A model using texture features to differentiate the nature of thyroid nodules on sonography. J. Ultrasound Med. 34, 1753–1760 (2015).

    PubMed  Google Scholar 

  122. Xu, L. et al. Computer-aided diagnosis systems in diagnosing malignant thyroid nodules on ultrasonography: a systematic review and meta-analysis. Eur. Thyroid. J. 9, 186–193 (2020).

    PubMed  Google Scholar 

  123. Shi, G. et al. Knowledge-guided synthetic medical image adversarial augmentation for ultrasonography thyroid nodule classification. Comput. Methods Prog. Biomed. 196, 105611 (2020).

    Google Scholar 

  124. Zhao, W. J., Fu, L. R., Huang, Z. M., Zhu, J. Q. & Ma, B. Y. Effectiveness evaluation of computer-aided diagnosis system for the diagnosis of thyroid nodules on ultrasound: a systematic review and meta-analysis. Medicine 98, e16379 (2019).

    PubMed  PubMed Central  Google Scholar 

  125. Buda, M. et al. Management of thyroid nodules seen on US images: deep learning may match performance of radiologists. Radiology 292, 695–701 (2019).

    PubMed  Google Scholar 

  126. Jeong, E. Y. et al. Computer-aided diagnosis system for thyroid nodules on ultrasonography: diagnostic performance and reproducibility based on the experience level of operators. Eur. Radiol. 29, 1978–1985 (2019).

    PubMed  Google Scholar 

  127. Sollini, M., Cozzi, L., Chiti, A. & Kirienko, M. Texture analysis and machine learning to characterize suspected thyroid nodules and differentiated thyroid cancer: where do we stand? Eur. J. Radiol. 99, 1–8 (2018).

    PubMed  Google Scholar 

  128. Choi, Y. J. et al. A computer-aided diagnosis system using artificial intelligence for the diagnosis and characterization of thyroid nodules on ultrasound: initial clinical assessment. Thyroid 27, 546–552 (2017).

    PubMed  Google Scholar 

  129. Daniels, K. et al. Machine learning by ultrasonography for genetic risk stratification of thyroid nodules. JAMA Otolaryngol. Head Neck Surg. 146, 36–41 (2020).

    PubMed  Google Scholar 

  130. Kosilek, R. P. et al. Diagnostic use of facial image analysis software in endocrine and genetic disorders: review, current results and future perspectives. Eur. J. Endocrinol. 173, M39–44 (2015).

    CAS  PubMed  Google Scholar 

  131. Kong, X., Gong, S., Su, L., Howard, N. & Kong, Y. Automatic detection of acromegaly from facial photographs using machine learning methods. EBioMedicine 27, 94–102 (2018). This study evaluates multiple machine learning and deep learning models to differentiate patients with acromegaly from facial photographs, with the top-performing ensemble model achieving a diagnostic accuracy that was on par with that of specialists and superior to that of primary care physicians.

    PubMed  Google Scholar 

  132. Schneider, H. J. et al. A novel approach to the detection of acromegaly: accuracy of diagnosis by automatic face classification. J. Clin. Endocrinol. Metab. 96, 2074–2080 (2011).

    CAS  PubMed  Google Scholar 

  133. Kosilek, R. P. et al. Automatic face classification of Cushing’s syndrome in women — a novel screening approach. Exp. Clin. Endocrinol. Diabetes 121, 561–564 (2013).

    CAS  PubMed  Google Scholar 

  134. Popp, K. H. et al. Computer vision technology in the differential diagnosis of Cushing’s syndrome. Exp. Clin. Endocrinol. Diabetes 127, 685–690 (2019).

    CAS  PubMed  Google Scholar 

  135. Dal, J. et al. Disease control and gender predict the socioeconomic effects of acromegaly: a nationwide cohort study. J. Clin. Endocrinol. Metab. 105, 2975–2982 (2020).

    Google Scholar 

  136. Gkourogianni, A. et al. Pediatric Cushing disease: disparities in disease severity and outcomes in the Hispanic and African-American populations. Pediatr. Res. 82, 272–277 (2017).

    PubMed  PubMed Central  Google Scholar 

  137. Lambin, P. et al. Radiomics: the bridge between medical imaging and personalized medicine. Nat. Rev. Clin. Oncol. 14, 749–762 (2017).

    PubMed  Google Scholar 

  138. Mongan, J., Moy, L. & Kahn, C. E. Jr Checklist for artificial intelligence in medical imaging (CLAIM): a guide for authors and reviewers. Radiol. Artif. Intell. 2, e200029 (2020).

    PubMed  PubMed Central  Google Scholar 

  139. Liu, X., Cruz Rivera, S., Moher, D., Calvert, M. J. & Denniston, A. K. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nat. Med. 26, 1364–1374 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  140. Miotto, R., Wang, F., Wang, S., Jiang, X. & Dudley, J. T. Deep learning for healthcare: review, opportunities and challenges. Brief. Bioinform. 19, 1236–1246 (2018).

    PubMed  Google Scholar 

  141. Topol, E. J. High-performance medicine: the convergence of human and artificial intelligence. Nat. Med. 25, 44–56 (2019).

    CAS  PubMed  Google Scholar 

  142. Wang, F., Kaushal, R. & Khullar, D. Should health care demand interpretable artificial intelligence or accept “Black Box” Medicine? Ann. Intern. Med. 172, 59–60 (2019).

    PubMed  Google Scholar 

  143. Murdoch, W. J., Singh, C., Kumbier, K., Abbasi-Asl, R. & Yu, B. Definitions, methods, and applications in interpretable machine learning. Proc. Natl Acad. Sci. USA 116, 22071–22080 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  144. Reyes, M. et al. On the interpretability of artificial intelligence in radiology: challenges and opportunities. Radiol. Artif. Intell. 2, e190043 (2020).

    PubMed  PubMed Central  Google Scholar 

  145. Chattopadhay, A., Sarkar, A., Howlader, P. & Balasubramanian, V. N. Grad-CAM++: generalized gradient-based visual explanations for deep convolutional networks. IEEE Winter Conf. Appl. Comput. Vis. https://doi.org/10.1109/WACV.2018.00097 (2018).

    Article  Google Scholar 

  146. Ribeiro, M. T., Singh, S. & Guestrin, C. Why Should I Trust You?: Explaining the Predictions of Any Classifier. in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1135-1144 (Association for Computing Machinery, 2016).

  147. Akkus, Z. et al. Reduction of Unnecessary Thyroid Biopsies using Deep Learning. Vol. 10949 MI (SPIE, 2019).

  148. Pereira, S. et al. Enhancing interpretability of automatically extracted machine learning features: application to a RBM-Random Forest system on brain lesion segmentation. Med. Image Anal. 44, 228–244 (2018).

    PubMed  Google Scholar 

  149. Natekar, P., Kori, A. & Krishnamurthi, G. Demystifying brain tumor segmentation networks: interpretability and uncertainty analysis. Front. Comput. Neurosci. 14, 6 (2020).

    PubMed  PubMed Central  Google Scholar 

  150. Philbrick, K. A. et al. What does deep learning see? Insights from a classifier trained to predict contrast enhancement phase from CT images. Am. J. Roentgenol. 211, 1184–1193 (2018).

    Google Scholar 

  151. Thomas, J. & Haertling, T. AIBx, artificial intelligence model to risk stratify thyroid nodules. Thyroid 30, 878–884 (2020).

    PubMed  Google Scholar 

  152. Gallego-Ortiz, C. & Martel, A. L. Using quantitative features extracted from T2-weighted MRI to improve breast MRI computer-aided diagnosis (CAD). PLoS One 12, e0187501 (2017).

    PubMed  PubMed Central  Google Scholar 

  153. Gale, W., Oakden-Rayner, L., Carneiro, G., Palmer, L. J. & Bradley, A. P. Producing radiologist-quality reports for interpretable deep learning. IEEE Int. Symp. Biomed. Imaging https://doi.org/10.1109/ISBI.2019.8759236 (2019).

    Article  Google Scholar 

  154. Wu, F. et al. A new coronavirus associated with human respiratory disease in China. Nature 579, 265–269 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  155. Shuja, J., Alanazi, E., Alasmary, W. & Alashaikh, A. COVID-19 open source data sets: a comprehensive survey. Appl. Intell. 51, 1296–1325 (2021).

    Google Scholar 

  156. Bai, H. X. & Thomasian, N. M. RICORD: a precedent for open AI in COVID-19 image analytics. Radiology 299, E219–E220 (2021).

    PubMed  Google Scholar 

  157. Marcus, D. S., Olsen, T. R., Ramaratnam, M. & Buckner, R. L. The extensible neuroimaging archive toolkit: an informatics platform for managing, exploring, and sharing neuroimaging data. Neuroinformatics 5, 11–34 (2007).

    PubMed  Google Scholar 

  158. Dao, T. et al. A kernel theory of modern data augmentation. Proc. Mach. Learn. Res. 97, 1528–1537 (2019).

    PubMed  PubMed Central  Google Scholar 

  159. Hussain, Z., Gimenez, F., Yi, D. & Rubin, D. Differential data augmentation techniques for medical imaging classification tasks. AMIA Annu. Symp. Proc. 2017, 979–984 (2017).

    PubMed  Google Scholar 

  160. Deepak, S. & Ameer, P. M. Brain tumor classification using deep CNN features via transfer learning. Comput. Biol. Med. 111, 103345 (2019).

    CAS  PubMed  Google Scholar 

  161. Shin, H. C. et al. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imaging 35, 1285–1298 (2016).

    PubMed  Google Scholar 

  162. Sheller, M. J. et al. Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data. Sci. Rep. 10, 12598 (2020).

    PubMed  PubMed Central  Google Scholar 

  163. Chang, K. et al. Distributed deep learning networks among institutions for medical imaging. J. Am. Med. Inf. Assoc. 25, 945–954 (2018). This paper provides a helpful overview and empirical demonstration of distributed learning techniques for multi-institutional collaborative deep learning model training.

    Google Scholar 

  164. Sheller, M. J., Reina, G. A., Edwards, B., Martin, J. & Bakas, S. Multi-institutional deep learning modeling without sharing patient data: a feasibility study on brain tumor segmentation. Brainlesion 11383, 92–104 (2019).

    PubMed  PubMed Central  Google Scholar 

  165. Biamonte, J. et al. Quantum machine learning. Nature 549, 195–202 (2017).

    CAS  PubMed  Google Scholar 

  166. Arute, F. et al. Quantum supremacy using a programmable superconducting processor. Nature 574, 505–510 (2019).

    CAS  PubMed  Google Scholar 

  167. Princeton University Center for Information Technology Policy. Implications of quantum computing for encryption policy. CEIP. https://carnegieendowment.org/2019/04/25/implications-of-quantum-computing-for-encryption-policy-pub-78985 (2019).

  168. Smets, E. et al. Large-scale wearable data reveal digital phenotypes for daily-life stress detection. NPJ Digital Med. 1, 67 (2018).

    Google Scholar 

  169. Tuncer, S. A. & Alkan, A. Segmentation of thyroid nodules with K-means algorithm on mobile devices. IEEE Int. Symp. Biomed. Imaging https://doi.org/10.1109/CINTI.2015.7382947 (2015).

    Article  Google Scholar 

  170. Ma, J. et al. Efficient deep learning architecture for detection and recognition of thyroid nodules. Comput. Intell. Neurosci. 2020, 1242781 (2020).

    PubMed  PubMed Central  Google Scholar 

  171. Poudel, P., Illanes, A., Sheet, D. & Friebe, M. Evaluation of commonly used algorithms for thyroid ultrasound images segmentation and improvement using machine learning approaches. J. Healthc. Eng. 2018, 8087624 (2018).

    PubMed  PubMed Central  Google Scholar 

  172. Shin, H. C., Orton, M. R., Collins, D. J., Doran, S. J. & Leach, M. O. Stacked autoencoders for unsupervised feature learning and multiple organ detection in a pilot study using 4D patient data. IEEE Trans. Pattern Anal. Mach. Intell. 35, 1930–1943 (2013).

    PubMed  Google Scholar 

  173. Song, J. et al. Ultrasound image analysis using deep learning algorithm for the diagnosis of thyroid nodules. Medicine 98, e15133 (2019).

    PubMed  PubMed Central  Google Scholar 

  174. Wang, H. et al. Machine learning-based multiparametric MRI radiomics for predicting the aggressiveness of papillary thyroid carcinoma. Eur. J. Radiol. 122, 108755 (2020). This study found a machine learning pipeline with LASSO for feature selection with a Gradient Boosting Classifier for classification that was superior to clinical characteristics in terms of preoperatively differentiating aggressive versus non-aggressive papillary thyroid carcinoma.

    PubMed  Google Scholar 

  175. Haugen, B. R. et al. 2015 American Thyroid Association Management Guidelines for Adult Patients with Thyroid Nodules and Differentiated Thyroid Cancer: The American Thyroid Association Guidelines Task Force on Thyroid Nodules and Differentiated Thyroid Cancer. Thyroid 26, 1–133 (2016).

    PubMed  PubMed Central  Google Scholar 

  176. Yi, T. et al. DICOM Image Analysis and Archive (DIANA): an open-source system for clinical AI applications. J. Digit. Imaging https://doi.org/10.1007/s10278-021-00488-5 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  177. Perseguers, S., Lewenstein, M., Acín, A. & Cirac, J. I. Quantum random networks. Nat. Phys. 6, 539–543 (2010).

    CAS  Google Scholar 

  178. Biamonte, J., Faccin, M. & De Domenico, M. Complex networks from classical to quantum. Commun. Phys. 2, 53 (2019).

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Warren Alpert Medical School of Brown University, Providence, RI, USA

    Nicole M. Thomasian

  2. Department of Imaging & Imaging Sciences, Johns Hopkins University School of Medicine, Baltimore, MD, USA

    Ihab R. Kamel & Harrison X. Bai

Authors
  1. Nicole M. Thomasian
    View author publications

    Search author on:PubMed Google Scholar

  2. Ihab R. Kamel
    View author publications

    Search author on:PubMed Google Scholar

  3. Harrison X. Bai
    View author publications

    Search author on:PubMed Google Scholar

Contributions

The authors contributed equally to all aspects of the article.

Corresponding author

Correspondence to Harrison X. Bai.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information

Nature Reviews Endocrinology thanks J. Thomas and J. Wang for their contribution to the peer review of this work.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Related links

International Agency for Research on Cancer: https://gco.iarc.fr/

UK Biobank: https://www.ukbiobank.ac.uk/imaging-data/

US National Institute of Health Cancer Imaging Archive: https://www.cancerimagingarchive.net

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Thomasian, N.M., Kamel, I.R. & Bai, H.X. Machine intelligence in non-invasive endocrine cancer diagnostics. Nat Rev Endocrinol 18, 81–95 (2022). https://doi.org/10.1038/s41574-021-00543-9

Download citation

  • Accepted:

  • Published:

  • Issue date:

  • DOI: https://doi.org/10.1038/s41574-021-00543-9

This article is cited by

Search

Advanced search

Quick links

Nature Briefing: Cancer

Sign up for the Nature Briefing: Cancer newsletter — what matters in cancer research, free to your inbox weekly.

Get what matters in cancer research, free to your inbox weekly. Sign up for Nature Briefing: Cancer