Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Review Article
  • Published:

Image-based profiling for drug discovery: due for a machine-learning upgrade?

Nature Reviews Drug Discovery volume 20, pages 145–159 (2021)Cite this article

Subjects

Abstract

Image-based profiling is a maturing strategy by which the rich information present in biological images is reduced to a multidimensional profile, a collection of extracted image-based features. These profiles can be mined for relevant patterns, revealing unexpected biological activity that is useful for many steps in the drug discovery process. Such applications include identifying disease-associated screenable phenotypes, understanding disease mechanisms and predicting a drug’s activity, toxicity or mechanism of action. Several of these applications have been recently validated and have moved into production mode within academia and the pharmaceutical industry. Some of these have yielded disappointing results in practice but are now of renewed interest due to improved machine-learning strategies that better leverage image-based information. Although challenges remain, novel computational technologies such as deep learning and single-cell methods that better capture the biological information in images hold promise for accelerating drug discovery.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to the full article PDF.

USD 39.95

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Image-based profiling.

Similar content being viewed by others

References

  1. Haasen, D. et al. How phenotypic screening influenced drug discovery: lessons from five years of practice. Assay Drug Dev. Technol. 15, 239–246 (2017). This primer describes the hype and eventual more nuanced positioning of phenotypic screening, providing a pharmaceutical industry perspective as machine learning is ready to spark a new wave of opportunities in this space.

    CAS  PubMed  Google Scholar 

  2. Singh, S., Carpenter, A. E. & Genovesio, A. Increasing the content of high-content screening: an overview. J. Biomol. Screen. 19, 640–650 (2014).

    PubMed  PubMed Central  Google Scholar 

  3. Dorval, T., Chanrion, B., Cattin, M.-E. & Stephan, J. P. Filling the drug discovery gap: is high-content screening the missing link? Curr. Opin. Pharmacol. 42, 40–45 (2018).

    CAS  PubMed  Google Scholar 

  4. Boutros, M., Heigwer, F. & Laufer, C. Microscopy-based high-content screening. Cell 163, 1314–1325 (2015).

    CAS  PubMed  Google Scholar 

  5. Caicedo, J. C., Singh, S. & Carpenter, A. E. Applications in image-based profiling of perturbations. Curr. Opin. Biotechnol. 39, 134–142 (2016).

    CAS  PubMed  Google Scholar 

  6. Herholt, A., Galinski, S., Geyer, P. E., Rossner, M. J. & Wehr, M. C. Multiparametric assays for accelerating early drug discovery. Trends Pharmacol. Sci. https://doi.org/10.1016/j.tips.2020.02.005 (2020).

    Article  PubMed  Google Scholar 

  7. David, L. et al. Applications of deep-learning in exploiting large-scale and heterogeneous compound data in industrial pharmaceutical research. Front. Pharmacol. 10, 1303 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  8. Stinson, S. F. et al. Morphological and immunocytochemical characteristics of human tumor cell lines for use in a disease-oriented anticancer drug screen. Anticancer. Res. 12, 1035–1053 (1992).

    CAS  PubMed  Google Scholar 

  9. Merget, B., Turk, S., Eid, S., Rippmann, F. & Fulle, S. Profiling prediction of kinase inhibitors: toward the virtual assay. J. Med. Chem. 60, 474–485 (2017).

    CAS  PubMed  Google Scholar 

  10. Bowes, J. et al. Reducing safety-related drug attrition: the use of in vitro pharmacological profiling. Nat. Rev. Drug Discov. 11, 909–922 (2012).

    CAS  PubMed  Google Scholar 

  11. Kurita, K. L. & Linington, R. G. Connecting phenotype and chemotype: high-content discovery strategies for natural products research. J. Nat. Prod. 78, 587–596 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  12. Litichevskiy, L. et al. A library of phosphoproteomic and chromatin signatures for characterizing cellular responses to drug perturbations. Cell Syst. 6, 424–443.e7 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  13. Bouzekri, A., Esch, A. & Ornatsky, O. Multidimensional profiling of drug‐treated cells by imaging mass cytometry. FEBS Open Bio 9, 1652–1669 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  14. Zampieri, M., Sekar, K., Zamboni, N. & Sauer, U. Frontiers of high-throughput metabolomics. Curr. Opin. Chem. Biol. 36, 15–23 (2017).

    CAS  PubMed  Google Scholar 

  15. Dubuis, S., Ortmayr, K. & Zampieri, M. A framework for large-scale metabolome drug profiling links coenzyme A metabolism to the toxicity of anti-cancer drug dichloroacetate. Commun. Biol. 1, 101 (2018).

    PubMed  PubMed Central  Google Scholar 

  16. Ye, C. et al. DRUG-seq for miniaturized high-throughput transcriptome profiling in drug discovery. Nat. Commun. 9, 4307 (2018).

    PubMed  PubMed Central  Google Scholar 

  17. Keenan, A. B. et al. Connectivity mapping: methods and applications. Annu. Rev. Biomed. Data Sci. 2, 69–92 (2019).

    Google Scholar 

  18. Subramanian, A. et al. A next generation connectivity map: L1000 platform and the first 1,000,000 profiles. Cell 171, 1437–1452.e17 (2017).

    CAS  PubMed  PubMed Central  Google Scholar 

  19. Corsello, S. M. et al. Discovering the anticancer potential of non-oncology drugs by systematic viability profiling. Nat. Cancer 1, 235–248 (2020).

    PubMed  PubMed Central  Google Scholar 

  20. Bray, M.-A. et al. Cell Painting, a high-content image-based assay for morphological profiling using multiplexed fluorescent dyes. Nat. Protoc. 11, 1757–1774 (2016). This step-by-step protocol article details Cell Painting, the most popular image-based profiling assay; it combines six stains to label eight cell components in a straightforward microscopy assay.

    CAS  PubMed  PubMed Central  Google Scholar 

  21. Lapins, M. & Spjuth, O. Evaluation of gene expression and phenotypic profiling data as quantitative descriptors for predicting drug targets and mechanisms of action. Preprint at bioRxiv https://doi.org/10.1101/580654 (2019).

    Article  Google Scholar 

  22. Wawer, M. J. et al. Toward performance-diverse small-molecule libraries for cell-based phenotypic screening using multiplexed high-dimensional profiling. Proc. Natl Acad. Sci. USA https://doi.org/10.1073/pnas.1410933111 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  23. Weigle, S., Martin, E., Voegtle, A., Wahl, B. & Schuler, M. Primary cell-based phenotypic assays to pharmacologically and genetically study fibrotic diseases in vitro. J. Biol. Methods 6, e115 (2019).

    PubMed  PubMed Central  Google Scholar 

  24. Johnson, K. et al. A stem cell-based approach to cartilage repair. Science 336, 717–721 (2012).

    CAS  PubMed  Google Scholar 

  25. Simm, J. et al. Repurposing high-throughput image assays enables biological activity prediction for drug discovery. Cell Chem. Biol. 25, 611–618.e3 (2018). This study applies machine learning to microscopy images designed in the context of a specific screening campaign to inform on the activity of compounds in a broad spectrum of biological processes; this has applications in early drug discovery.

    CAS  PubMed  PubMed Central  Google Scholar 

  26. Méndez-Lucio, O., Zapata, P. A. M., Wichard, J., Rouquié, D. & Clevert, D.-A. Cell morphology-guided de novo hit design by conditioning generative adversarial networks on phenotypic image features. Preprint at ChemRxiv https://doi.org/10.26434/chemrxiv.11594067 (2020). This study proposes an approach to image-based lead optimization: a generative adversarial network is trained on small-molecule structures with and without matching image-based profiles to generate new small-molecule structures that mimic the morphological profiles induced by overexpression of 12 individual genes.

    Article  Google Scholar 

  27. Mullard, A. Machine learning brings cell imaging promises into focus. Nat. Rev. Drug Discov. 18, 653–655 (2019).

    CAS  PubMed  Google Scholar 

  28. Mullard, A. Daphne Koller. Nat. Rev. Drug Discov. 18, 576–577 (2019).

    CAS  PubMed  Google Scholar 

  29. Feldman, D. et al. Optical pooled screens in human. Cells. Cell 179, 787–799.e17 (2019).

    CAS  PubMed  Google Scholar 

  30. Cooper, S., Sadok, A., Bousgouni, V. & Bakal, C. Apolar and polar transitions drive the conversion between amoeboid and mesenchymal shapes in melanoma cells. Mol. Biol. Cell 26, 4163–4170 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  31. Eismann, B. et al. Automated 3D light-sheet screening with high spatiotemporal resolution reveals mitotic phenotypes. J. Cell Sci. 133, jcs245043 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  32. Christiansen, E. M. et al. In silico labeling: predicting fluorescent labels in unlabeled images. Cell 173, 792–803.e19 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  33. Ounkomol, C., Seshamani, S., Maleckar, M. M., Collman, F. & Johnson, G. R. Label-free prediction of three-dimensional fluorescence images from transmitted-light microscopy. Nat. Methods 15, 917–920 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  34. Grys, B. T. et al. Machine learning and computer vision approaches for phenotypic profiling. J. Cell Biol. 216, 65–71 (2017).

    CAS  PubMed  PubMed Central  Google Scholar 

  35. Caicedo, J. C. et al. Data-analysis strategies for image-based cell profiling. Nat. Methods 14, 849–863 (2017). Members of the CytoData Society across most laboratories in the field of image-based profiling come together to define best practices for informatics; remarkably, these are still relatively up to date, with the exception of deep learning advancements.

    CAS  PubMed  PubMed Central  Google Scholar 

  36. Price, J. H. et al. Advances in molecular labeling, high throughput imaging and machine intelligence portend powerful functional cellular biochemistry tools. J. Cell. Biochem. Suppl. 39, 194–210 (2002).

    PubMed  Google Scholar 

  37. Perlman, Z. E. et al. Multidimensional drug profiling by automated microscopy. Science 306, 1194–1198 (2004). This article arguably launched the field of image-based profiling, as it presents the high-dimensional analysis of images of cells in the presence of chemical perturbations.

    CAS  PubMed  Google Scholar 

  38. Abraham, V. C., Taylor, D. L. & Haskins, J. R. High content screening applied to large-scale cell biology. Trends Biotechnol. 22, 15–22 (2004).

    CAS  PubMed  Google Scholar 

  39. Michael Ando, D., McLean, C. Y. & Berndl, M. Improving phenotypic measurements in high-content imaging screens. Preprint at bioRxiv https://doi.org/10.1101/161422 (2017).

    Article  Google Scholar 

  40. van der Maaten, L. Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008).

    Google Scholar 

  41. McInnes, L., Healy, J. & Melville, J. UMAP: uniform manifold approximation and projection for dimension reduction. Preprint at arXiv https://arxiv.org/abs/1802.03426 (2018).

  42. Pearson, K. LIII. On lines and planes of closest fit to systems of points in space. Lond. Edinb. Dublin Philos. Mag. J. Sci. 2, 559–572 (1901).

    Google Scholar 

  43. Kang, J. et al. Improving drug discovery with high-content phenotypic screens by systematic selection of reporter cell lines. Nat. Biotechnol. 34, 70–77 (2016).

    CAS  PubMed  Google Scholar 

  44. Cox, M. J. et al. Tales of 1,008 small molecules: phenomic profiling through live-cell imaging in a panel of reporter cell lines. Sci. Rep. 10, 13262 (2020). This study quantifies the improvement in the ability of image-based profiling using Cell Painting to distinguish compound MOAs in 15 cell lines versus the best single cell line (approximately twofold); use of data from all 15 cell lines distinguished only about half of the MOAs.

    CAS  PubMed  PubMed Central  Google Scholar 

  45. Scheeder, C., Heigwer, F. & Boutros, M. Machine learning and image-based profiling in drug discovery. Curr. Opin. Syst. Biol. 10, 43–52 (2018).

    PubMed  PubMed Central  Google Scholar 

  46. Lu, A. X., Kraus, O. Z., Cooper, S. & Moses, A. M. Learning unsupervised feature representations for single cell microscopy images with paired cell inpainting. PLoS Comput. Biol. 15, e1007348 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  47. Zhou, Z.-H. A brief introduction to weakly supervised learning. Natl Sci. Rev. 5, 44–53 (2018).

    Google Scholar 

  48. Caicedo, J. C., McQuin, C., Goodman, A., Singh, S. & Carpenter, A. E. Weakly supervised learning of single-cell feature embeddings. Proc IEEE Comput. Soc. Conf. Comput Vis. Pattern Recognit. https://doi.org/10.1109/CVPR.2018.00970 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  49. Teschendorff, A. E. Avoiding common pitfalls in machine learning omic data science. Nat. Mater. 18, 422–427 (2019).

    CAS  PubMed  Google Scholar 

  50. Riley, P. Three pitfalls to avoid in machine learning. Nature 572, 27–29 (2019).

    CAS  PubMed  Google Scholar 

  51. Pegoraro, G. & Misteli, T. High-throughput imaging for the discovery of cellular mechanisms of disease. Trends Genet. 33, 604–615 (2017).

    CAS  PubMed  PubMed Central  Google Scholar 

  52. Cataldo, A. M. et al. Abnormalities in mitochondrial structure in cells from patients with bipolar disorder. Am. J. Pathol. 177, 575–585 (2010).

    PubMed  PubMed Central  Google Scholar 

  53. Blanchet, L. et al. Quantifying small molecule phenotypic effects using mitochondrial morpho-functional fingerprinting and machine learning. Sci. Rep. 5, 8035 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  54. Hung, C. L.-K. et al. A patient-derived cellular model for Huntington’s disease reveals phenotypes at clinically relevant CAG lengths. Mol. Biol. Cell 29, 2809–2820 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  55. Smith, I. et al. Evaluation of RNAi and CRISPR technologies by large-scale gene expression profiling in the Connectivity Map. PLoS Biol. 15, e2003213 (2017).

    PubMed  PubMed Central  Google Scholar 

  56. Gibson, C. C. et al. Strategy for identifying repurposed drugs for the treatment of cerebral cavernous malformation. Circulation 131, 289–299 (2015).

    CAS  PubMed  Google Scholar 

  57. Jiao, Y. et al. Discovering metabolic disease gene interactions by correlated effects on cellular morphology. Mol. Metab. 24, 108–119 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  58. Thyme, S. B. et al. Phenotypic landscape of schizophrenia-associated genes defines candidates and their shared functions. Cell 177, 478–491.e20 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  59. Rohban, M. H. et al. Systematic morphological profiling of human gene and allele function via cell painting. eLife 6, e24060 (2017).

    PubMed  PubMed Central  Google Scholar 

  60. Berger, A. H. et al. High-throughput phenotyping of lung cancer somatic mutations. Cancer Cell 32, 884 (2017).

    CAS  PubMed  Google Scholar 

  61. Ben-David, U. et al. Genetic and transcriptional evolution alters cancer cell line drug response. Nature 560, 325–330 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  62. Dawson, J. C. et al. A synergistic anti-cancer FAK and HDAC inhibitor combination discovered by a novel chemical-genetic high-content phenotypic screen. Mol. Cancer Ther. https://doi.org/10.1101/590802 (2020).

    Article  PubMed  Google Scholar 

  63. Papanikolopoulou, K., Mudher, A. & Skoulakis, E. An assessment of the translational relevance of Drosophila in drug discovery. Expert Opin. Drug Discov. 14, 303–313 (2019).

    CAS  PubMed  Google Scholar 

  64. Costa, B., Estrada, M. F., Mendes, R. V. & Fior, R. Zebrafish avatars towards personalized medicine-a comparative review between avatar models. Cells 9, 293 (2020).

    CAS  PubMed Central  Google Scholar 

  65. Astone, M., Dankert, E. N., Alam, S. K. & Hoeppner, L. H. Fishing for cures: the alLURE of using zebrafish to develop precision oncology therapies. NPJ Precis. Oncol. 1, 39 (2017).

    PubMed  PubMed Central  Google Scholar 

  66. Vashi, N. & Justice, M. J. Treating Rett syndrome: from mouse models to human therapies. Mamm. Genome 30, 90–110 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  67. Dar, A. C., Das, T. K., Shokat, K. M. & Cagan, R. L. Chemical genetic discovery of targets and anti-targets for cancer polypharmacology. Nature 486, 80–84 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  68. Rihel, J. et al. Zebrafish behavioral profiling links drugs to biological targets and rest/wake regulation. Science 327, 348–351 (2010).

    CAS  PubMed  PubMed Central  Google Scholar 

  69. Bruni, G. et al. Zebrafish behavioral profiling identifies multitarget antipsychotic-like compounds. Nat. Chem. Biol. 12, 559–566 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  70. Jordi, J. et al. High-throughput screening for selective appetite modulators: a multibehavioral and translational drug discovery strategy. Sci. Adv. 4, eaav1966 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  71. McCarroll, M. N. et al. Zebrafish behavioural profiling identifies GABA and serotonin receptor ligands related to sedation and paradoxical excitation. Nat. Commun. 10, 4078 (2019).

    PubMed  PubMed Central  Google Scholar 

  72. Kümmel, A. et al. Differentiation and visualization of diverse cellular phenotypic responses in primary high-content screening. J. Biomol. Screen. 17, 843–849 (2012).

    PubMed  Google Scholar 

  73. Loo, L.-H., Wu, L. F. & Altschuler, S. J. Image-based multivariate profiling of drug responses from single cells. Nat. Methods 4, 445–453 (2007).

    CAS  PubMed  Google Scholar 

  74. Vial, M.-L. et al. A grand challenge. 2. Phenotypic profiling of a natural product library on Parkinson’s patient-derived cells. J. Nat. Prod. 79, 1982–1989 (2016).

    CAS  PubMed  Google Scholar 

  75. Caie, P. D. et al. High-content phenotypic profiling of drug response signatures across distinct cancer cells. Mol. Cancer Ther. 9, 1913–1926 (2010).

    CAS  PubMed  Google Scholar 

  76. Hughes, R. E. et al. High-content phenotypic profiling in esophageal adenocarcinoma identifies selectively active pharmacological classes of drugs for repurposing and chemical starting points for novel drug discovery. SLAS Discov. https://doi.org/10.1177/2472555220917115 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  77. Gerry, C. J. et al. Real-time biological annotation of synthetic compounds. J. Am. Chem. Soc. 138, 8920–8927 (2016). This study launched many further examples using Cell Painting interactively with synthetic chemistry for medicinal chemistry and SAR determination.

    CAS  PubMed  PubMed Central  Google Scholar 

  78. Nelson, S. D. Jr. Wawer, M. J. & Schreiber, S. L. Divergent synthesis and real-time biological annotation of optically active tetrahydrocyclopenta[c]pyranone derivatives. Org. Lett. 18, 6280–6283 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  79. Melillo, B. et al. Synergistic effects of stereochemistry and appendages on the performance diversity of a collection of synthetic compounds. J. Am. Chem. Soc. 140, 11784–11790 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  80. Zimmermann, S. et al. A scaffold-diversity synthesis of biologically intriguing cyclic sulfonamides. Chemistry 25, 15498–15503 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  81. Gaulton, A. et al. The ChEMBL database in 2017. Nucleic Acids Res. 45, D945–D954 (2017).

    CAS  PubMed  Google Scholar 

  82. Bray, M.-A. et al. A dataset of images and morphological profiles of 30,000 small-molecule treatments using the Cell Painting assay. Gigascience https://doi.org/10.1093/gigascience/giw014 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  83. Hofmarcher, M., Rumetshofer, E., Clevert, D.-A., Hochreiter, S. & Klambauer, G. Accurate prediction of biological assays with high-throughput microscopy images and convolutional networks. J. Chem. Inf. Model. 59, 1163–1171 (2019). This article describes how combining convolutional neural networks with images of broad, generic stains boosts biological information extraction from microscopy images.

    CAS  PubMed  Google Scholar 

  84. Hwang, T. J. et al. Failure of investigational drugs in late-stage clinical development and publication of trial results. JAMA Intern. Med. 176, 1826–1833 (2016).

    PubMed  Google Scholar 

  85. Thomas, R. S. et al. The next generation blueprint of computational toxicology at the US Environmental Protection Agency. Toxicol. Sci. 169, 317–332 (2019).

    CAS  PubMed  Google Scholar 

  86. Paul Friedman, K. et al. Utility of in vitro bioactivity as a lower bound estimate of in vivo adverse effect levels and in risk-based prioritization. Toxicol. Sci. 173, 202–225 (2020).

    PubMed  Google Scholar 

  87. Nyffeler, J. et al. Bioactivity screening of environmental chemicals using imaging-based high-throughput phenotypic profiling. Toxicol. Appl. Pharmacol. 389, 114876 (2020). This careful analysis describes the use of Cell Painting to report on various aspects of toxicity.

    CAS  PubMed  Google Scholar 

  88. Way, G. P. et al. Predicting cell health phenotypes using image-based morphology profiling. Preprint at bioRxiv https://doi.org/10.1101/2020.07.08.193938 (2020).

    Article  Google Scholar 

  89. Su, R., Xiong, S., Zink, D. & Loo, L.-H. High-throughput imaging-based nephrotoxicity prediction for xenobiotics with diverse chemical structures. Arch. Toxicol. 90, 2793–2808 (2016).

    CAS  PubMed  Google Scholar 

  90. Lee, J.-Y. J., Miller, J. A., Basu, S., Kee, T.-Z. V. & Loo, L.-H. Building predictive in vitro pulmonary toxicity assays using high-throughput imaging and artificial intelligence. Arch. Toxicol. 92, 2055–2075 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  91. Zhang, Q., Li, J., Peng, S., Zhang, Y. & Qiao, Y. Rosmarinic acid as a candidate in a phenotypic profiling cardio-/cytotoxicity cell model induced by doxorubicin. Molecules 25, 836 (2020).

    CAS  PubMed Central  Google Scholar 

  92. Yin, L. et al. High-content image-based single-cell phenotypic analysis for the testicular toxicity prediction induced by bisphenol A and its analogs bisphenol S, bisphenol AF, and tetrabromobisphenol A in a three-dimensional testicular cell co-culture model. Toxicol. Sci. 173, 313–335 (2020).

    CAS  PubMed  Google Scholar 

  93. Godinez, W. J. et al. Morphological deconvolution of beta-lactam polyspecificity in E. coli. ACS Chem. Biol. 14, 1217–1226 (2019).

    CAS  PubMed  Google Scholar 

  94. Tanaka, M. et al. An unbiased cell morphology–based screen for new, biologically active small molecules. PLoS Biol. 3, e128 (2005).

    PubMed  PubMed Central  Google Scholar 

  95. Lin, S. et al. Diversity focused semisyntheses of tetronate polyether ionophores. Preprint at ChemRxiv https://doi.org/10.26434/chemrxiv.8299715 (2019).

    Article  Google Scholar 

  96. Mayer, T. U. et al. Small molecule inhibitor of mitotic spindle bipolarity identified in a phenotype-based screen. Science 286, 971–974 (1999).

    CAS  PubMed  Google Scholar 

  97. Mechanism matters. Nat. Med. 16, 347 (2010).

  98. MacDonald, M. L. et al. Identifying off-target effects and hidden phenotypes of drugs in human cells. Nat. Chem. Biol. 2, 329–337 (2006).

    CAS  PubMed  Google Scholar 

  99. Schenone, M., Dančík, V., Wagner, B. K. & Clemons, P. A. Target identification and mechanism of action in chemical biology and drug discovery. Nat. Chem. Biol. 9, 232–240 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  100. Kubota, K., Funabashi, M. & Ogura, Y. Target deconvolution from phenotype-based drug discovery by using chemical proteomics approaches. Biochim. Biophys. Acta Proteins Proteom. 1867, 22–27 (2019).

    CAS  PubMed  Google Scholar 

  101. Woehrmann, M. H. et al. Large-scale cytological profiling for functional analysis of bioactive compounds. Mol. Biosyst. 9, 2604–2617 (2013).

    CAS  PubMed  Google Scholar 

  102. Slack, M. D., Martinez, E. D., Wu, L. F. & Altschuler, S. J. Characterizing heterogeneous cellular responses to perturbations. Proc. Natl Acad. Sci. USA 105, 19306–19311 (2008).

    CAS  PubMed  PubMed Central  Google Scholar 

  103. Ljosa, V. et al. Comparison of methods for image-based profiling of cellular morphological responses to small-molecule treatment. J. Biomol. Screen. 18, 1321–1329 (2013).

    CAS  PubMed  Google Scholar 

  104. Gustafsdottir, S. M. et al. Multiplex cytological profiling assay to measure diverse cellular States. PLoS ONE 8, e80999 (2013).

    PubMed  PubMed Central  Google Scholar 

  105. Gebre, A. A. et al. Profiling of the effects of antifungal agents on yeast cells based on morphometric analysis. FEMS Yeast Res. 15, fov040 (2015).

    PubMed  Google Scholar 

  106. Futamura, Y. et al. Morphobase, an encyclopedic cell morphology database, and its use for drug target identification. Chem. Biol. 19, 1620–1630 (2012).

    CAS  PubMed  Google Scholar 

  107. Reisen, F. et al. Linking phenotypes and modes of action through high-content screen fingerprints. ASSAY. Drug Dev. Technol. 13, 415–427 (2015).

    CAS  PubMed  Google Scholar 

  108. Laraia, L. et al. Image-based morphological profiling identifies a lysosomotropic, iron-sequestering autophagy inhibitor. Angew. Chem. Int. Ed. https://doi.org/10.1002/ange.201913712 (2020).

    Article  Google Scholar 

  109. Hansen, B. K. et al. Structure and function of the bacterial protein toxin phenomycin. Structure 28, 528–539.e9 (2020).

    CAS  PubMed  Google Scholar 

  110. Christoforow, A. et al. Design, synthesis, and phenotypic profiling of pyrano-furo-pyridone pseudo natural products. Angew. Chem. Int. Ed. 58, 14715–14723 (2019).

    CAS  Google Scholar 

  111. Peters, C. E. et al. Rapid inhibition profiling identifies a keystone target in the nucleotide biosynthesis pathway. ACS Chem. Biol. 13, 3251–3258 (2018).

    CAS  PubMed  Google Scholar 

  112. Schulze, C. J. et al. ‘Function-first’ lead discovery: mode of action profiling of natural product libraries using image-based screening. Chem. Biol. 20, 285–295 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  113. Ochoa, J. L., Bray, W. M., Lokey, R. S. & Linington, R. G. Phenotype-guided natural products discovery using cytological profiling. J. Nat. Prod. 78, 2242–2248 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  114. Kremb, S., Müller, C., Schmitt-Kopplin, P. & Voolstra, C. R. Bioactive potential of marine macroalgae from the central red sea (Saudi Arabia) assessed by high-throughput imaging-based phenotypic profiling. Mar. Drugs 15, 80 (2017).

    PubMed Central  Google Scholar 

  115. Boyd, J. C., Pinheiro, A., Nery, E. D., Reyal, F. & Walter, T. Domain-invariant features for mechanism of action prediction in a multi-cell-line drug screen. Bioinformatics https://doi.org/10.1093/bioinformatics/btz774 (2019).

    Article  PubMed Central  Google Scholar 

  116. Warchal, S. J. et al. High content phenotypic screening identifies serotonin receptor modulators with selective activity upon breast cancer cell cycle and cytokine signaling pathways. Bioorganic Medicinal Chem. 28, 115209 (2020).

    CAS  Google Scholar 

  117. Hight, S. K. et al. High-throughput functional annotation of natural products by integrated activity profiling. Pharmacol. Toxicol. https://doi.org/10.1101/748129 (2019).

    Article  Google Scholar 

  118. Kurita, K. L., Glassey, E. & Linington, R. G. Integration of high-content screening and untargeted metabolomics for comprehensive functional annotation of natural product libraries. Proc. Natl Acad. Sci. USA 112, 11999–12004 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  119. Nassiri, I. & McCall, M. N. Systematic exploration of cell morphological phenotypes associated with a transcriptomic query. Nucleic Acids Res. 46, e116 (2018).

    PubMed  PubMed Central  Google Scholar 

  120. Breinig, M., Klein, F. A., Huber, W. & Boutros, M. A chemical–genetic interaction map of small molecules using high-throughput imaging in cancer cells. Mol. Syst. Biol. 11, 846 (2015). This is a landmark study on the interaction of genes and compounds by use of isogenic cell lines, all mapped using image-based phenotypes.

    PubMed  PubMed Central  Google Scholar 

  121. Ohnuki, S., Oka, S., Nogami, S. & Ohya, Y. High-content, image-based screening for drug targets in yeast. PLoS ONE 5, e10177 (2010). This is a pioneering study in yeast demonstrating successful identification of small-molecule mimics of gene signatures (and vice versa for target identification) using image-based data.

    PubMed  PubMed Central  Google Scholar 

  122. Piotrowski, J. S. et al. Plant-derived antifungal agent poacic acid targets β-1,3-glucan. Proc. Natl Acad. Sci. USA 112, E1490–E1497 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  123. Iwaki, A., Ohnuki, S., Suga, Y., Izawa, S. & Ohya, Y. Vanillin inhibits translation and induces messenger ribonucleoprotein (mRNP) granule formation in saccharomyces cerevisiae: application and validation of high-content, image-based profiling. PLoS ONE 8, e61748 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  124. Sundaramurthy, V. et al. Integration of chemical and RNAi multiparametric profiles identifies triggers of intracellular mycobacterial killing. Cell Host Microbe 13, 129–142 (2013).

    CAS  PubMed  Google Scholar 

  125. Sundaramurthy, V. et al. Deducing the mechanism of action of compounds identified in phenotypic screens by integrating their multiparametric profiles with a reference genetic screen. Nat. Protoc. 9, 474–490 (2014).

    CAS  PubMed  Google Scholar 

  126. Eggert, U. S. et al. Parallel chemical genetic and genome-wide RNAi screens identify cytokinesis inhibitors and targets. PLoS Biol. 2, e379 (2004).

    PubMed  PubMed Central  Google Scholar 

  127. Singh, S. et al. Morphological profiles of RNAi-induced gene knockdown are highly reproducible but dominated by seed effects. PLoS ONE 10, e0131370 (2015).

    PubMed  PubMed Central  Google Scholar 

  128. Jinek, M. et al. A programmable dual-RNA-guided DNA endonuclease in adaptive bacterial immunity. Science 337, 816–821 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  129. Jackson, A. L. et al. Expression profiling reveals off-target gene regulation by RNAi. Nat. Biotechnol. 21, 635–637 (2003).

    CAS  PubMed  Google Scholar 

  130. Young, D. W. et al. Integrating high-content screening and ligand-target prediction to identify mechanism of action. Nat. Chem. Biol. 4, 59–68 (2008).

    CAS  PubMed  Google Scholar 

  131. Caldera, M. et al. Mapping the perturbome network of cellular perturbations. Nat. Commun. 10, 5140 (2019).

    PubMed  PubMed Central  Google Scholar 

  132. Fischer, B. et al. A map of directional genetic interactions in a metazoan cell. eLife 4, e05464 (2015).

    PubMed Central  Google Scholar 

  133. Yin, Z. et al. A screen for morphological complexity identifies regulators of switch-like transitions between discrete cell shapes. Nat. Cell Biol. 15, 860–871 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  134. Usaj, M. M. et al. Systematic genetics and single-cell imaging reveal widespread morphological pleiotropy and cell-to-cell variability. Mol. Syst. Biol. 16, e9243 (2020).

    PubMed  PubMed Central  Google Scholar 

  135. Chong, Y. T. et al. Yeast proteome dynamics from single cell imaging and automated analysis. Cell 161, 1413–1424 (2015).

    CAS  PubMed  Google Scholar 

  136. Neumann, B. et al. Phenotypic profiling of the human genome by time-lapse microscopy reveals cell division genes. Nature 464, 721–727 (2010).

    CAS  PubMed  PubMed Central  Google Scholar 

  137. Unadkat, H. V. et al. An algorithm-based topographical biomaterials library to instruct cell fate. Proc. Natl Acad. Sci. USA 108, 16565–16570 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  138. Ruan, X. et al. Image-derived models of cell organization changes during differentiation of PC12 cells. Preprint at bioRxiv https://doi.org/10.1101/522763 (2019).

    Article  Google Scholar 

  139. Doan, M. & Carpenter, A. E. Leveraging machine vision in cell-based diagnostics to do more with less. Nat. Mater. 18, 414–418 (2019).

    CAS  PubMed  Google Scholar 

  140. De Fauw, J. et al. Clinically applicable deep learning for diagnosis and referral in retinal disease. Nat. Med. 24, 1342–1350 (2018).

    PubMed  Google Scholar 

  141. Zhu, Y. et al. An image informatics pipeline for imaging mass cytometry to characterize the immune landscape in pre- and on-treatment immune therapy and its application in recurrent platinium-resistant epithelial ovarian cancer. in 2019 IEEE EMBS International Conference on Biomedical Health Informatics (BHI) 1–4 (IEEE, 2019).

  142. Mobadersany, P. et al. Predicting cancer outcomes from histology and genomics using convolutional networks. Proc. Natl Acad. Sci. USA 115, E2970–E2979 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  143. Jackson, H. W. et al. The single-cell pathology landscape of breast cancer. Nature 578, 615–620 (2020).

    CAS  PubMed  Google Scholar 

  144. Coudray, N. et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat. Med. 24, 1559–1567 (2018).

    CAS  PubMed  Google Scholar 

  145. Ash, J. T., Darnell, G., Munro, D. & Engelhardt, B. E. Joint analysis of gene expression levels and histological images identifies genes associated with tissue morphology. Preprint at bioRxiv https://doi.org/10.1101/458711 (2018).

    Article  Google Scholar 

  146. Quach, D. T., Sakoulas, G., Nizet, V., Pogliano, J. & Pogliano, K. Bacterial cytological profiling (BCP) as a rapid and accurate antimicrobial susceptibility testing method for staphylococcus aureus. EBioMedicine 4, 95–103 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  147. Dekkers, J. F. et al. A functional CFTR assay using primary cystic fibrosis intestinal organoids. Nat. Med. 19, 939–945 (2013).

    CAS  PubMed  Google Scholar 

  148. Snijder, B. et al. Image-based ex-vivo drug screening for patients with aggressive haematological malignancies: interim results from a single-arm, open-label, pilot study. Lancet Haematol. 4, e595–e606 (2017).

    PubMed  PubMed Central  Google Scholar 

  149. Irmisch, A. et al. The Tumor Profiler Study: integrated, multi-omic, functional tumor profiling for clinical decision support. Oncology https://doi.org/10.1101/2020.02.13.20017921 (2020).

    Article  Google Scholar 

  150. Betge, J. et al. Multiparametric phenotyping of compound effects on patient derived organoids. Preprint at bioRxiv https://doi.org/10.1101/660993 (2019).

    Article  Google Scholar 

  151. Chen, H., Engkvist, O., Wang, Y., Olivecrona, M. & Blaschke, T. The rise of deep learning in drug discovery. Drug Discov. Today 23, 1241–1250 (2018).

    PubMed  Google Scholar 

  152. Kraus, O. Z. & Frey, B. J. Computer vision for high content screening. Crit. Rev. Biochem. Mol. Biol. 51, 102–109 (2016).

    PubMed  Google Scholar 

  153. Moen, E. et al. Deep learning for cellular image analysis. Nat. Methods 16, 1233–1246 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  154. Chessel, A. & Carazo Salas, R. E. From observing to predicting single-cell structure and function with high-throughput/high-content microscopy. Essays Biochem. 63, 197–208 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  155. Wollmann, T. et al. GRUU-Net: integrated convolutional and gated recurrent neural network for cell segmentation. Med. Image Anal. 56, 68–79 (2019).

    CAS  PubMed  Google Scholar 

  156. Caicedo, J. C. et al. Nucleus segmentation across imaging experiments: the 2018 Data Science Bowl. Nat. Methods 16, 1247–1253 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  157. Jackson, P. T. et al. Phenotypic profiling of high throughput imaging screens with generic deep convolutional features. in 2019 16th International Conference on Machine Vision Applications (MVA) 1–4 (IEEE, 2019).

  158. McQuin, C. et al. CellProfiler 3.0: next-generation image processing for biology. PLoS Biol. 16, e2005970 (2018).

    PubMed  PubMed Central  Google Scholar 

  159. Eulenberg, P. et al. Reconstructing cell cycle and disease progression using deep learning. Nat. Commun. 8, 463 (2017).

    PubMed  PubMed Central  Google Scholar 

  160. Kensert, A., Harrison, P. J. & Spjuth, O. Transfer learning with deep convolutional neural networks for classifying cellular morphological changes. SLAS Discov. 24, 466–475 (2019).

    PubMed  PubMed Central  Google Scholar 

  161. Rohban, M. H., Abbasi, H. S., Singh, S. & Carpenter, A. E. Capturing single-cell heterogeneity via data fusion improves image-based profiling. Nat. Commun. 10, 2082 (2019).

    PubMed  PubMed Central  Google Scholar 

  162. Lim, N. & Pavlidis, P. Evaluation of connectivity map shows limited reproducibility in drug repositioning. Preprint at bioRxiv https://doi.org/10.1101/845693 (2019).

    Article  Google Scholar 

  163. Ellenberg, J. et al. A call for public archives for biological image data. Nat. Methods 15, 849–854 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  164. Swinney, D. C. The contribution of mechanistic understanding to phenotypic screening for first-in-class medicines. J. Biomol. Screen. 18, 1186–1192 (2013).

    PubMed  Google Scholar 

  165. Eder, J., Sedrani, R. & Wiesmann, C. The discovery of first-in-class drugs: origins and evolution. Nat. Rev. Drug Discov. 13, 577–587 (2014).

    CAS  PubMed  Google Scholar 

  166. Warchal, S. J., Unciti-Broceta, A. & Carragher, N. O. Next-generation phenotypic screening. Future Med. Chem. 8, 1331–1347 (2016).

    CAS  PubMed  Google Scholar 

  167. Swinney, D. C. & Anthony, J. How were new medicines discovered? Nat. Rev. Drug Discov. 10, 507–519 (2011).

    CAS  PubMed  Google Scholar 

  168. Vincent, F., Loria, P., Pregel, M. & Stanton, R. Developing predictive assays: the phenotypic screening ‘rule of 3’. Sci. Transl Med. https://doi.org/10.1126/scitranslmed.aab1201 (2015).

    Article  PubMed  Google Scholar 

  169. Lau, T. A., Bray, W. M. & Lokey, R. S. macrophage cytological profiling and anti-inflammatory drug discovery. Assay. Drug Dev. Technol. 17, 14–16 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  170. RxRx. https://www.rxrx.ai/.

  171. Lin, J.-R. et al. Highly multiplexed immunofluorescence imaging of human tissues and tumors using t-CyCIF and conventional optical microscopes. eLife 7, e31657 (2018).

    PubMed  PubMed Central  Google Scholar 

  172. Gerdes, M. J. et al. Highly multiplexed single-cell analysis of formalin-fixed, paraffin-embedded cancer tissue. Proc. Natl Acad. Sci. USA 110, 11982–11987 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  173. Bolognesi, M. M. et al. Multiplex staining by sequential immunostaining and antibody removal on routine tissue sections. J. Histochem. Cytochem. 65, 431–444 (2017).

    CAS  PubMed  PubMed Central  Google Scholar 

  174. Glass, G., Papin, J. A. & Mandell, J. W. SIMPLE: a sequential immunoperoxidase labeling and erasing method. J. Histochem. Cytochem. 57, 899–905 (2009).

    CAS  PubMed  PubMed Central  Google Scholar 

  175. Baharlou, H., Canete, N. P., Cunningham, A. L., Harman, A. N. & Patrick, E. Mass cytometry imaging for the study of human diseases-applications and data analysis strategies. Front. Immunol. 10, 2657 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  176. Rappez, L. et al. Spatial single-cell profiling of intracellular metabolomes in situ. Preprint at bioRxiv https://doi.org/10.1101/510222 (2019).

    Article  Google Scholar 

  177. Kang, Z. B. et al. Fluopack screening platform for unbiased cellular phenotype profiling. Sci. Rep. 10, 2097 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  178. Perez-Gomez, A. et al. A phenotypic caenorhabditis elegans screen identifies a selective suppressor of antipsychotic-induced hyperphagia. Nat. Commun. 9, 5272 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  179. Cassar, S. et al. Measuring drug absorption improves interpretation of behavioral responses in a larval zebrafish locomotor assay for predicting seizure liability. J. Pharmacol. Toxicol. Methods 88, 56–63 (2017).

    CAS  PubMed  Google Scholar 

  180. Becker, T., Caicedo, J. C., Singh, S., Weckmann, M. & Carpenter, A. E. Combining morphological and migration profiles of in vitro time-lapse data. in 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) 965–968 (IEEE, 2018).

  181. Qian, W. W. et al. Batch equalization with a generative adversarial network. Preprint at bioRxiv https://doi.org/10.1101/2020.02.07.939215 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  182. Rastelli, G. & Pinzi, L. Computational polypharmacology comes of age. Front. Pharmacol. 6, 157 (2015).

    PubMed  PubMed Central  Google Scholar 

  183. Proschak, E., Stark, H. & Merk, D. Polypharmacology by design: a medicinal chemist’s perspective on multitargeting compounds. J. Med. Chem. 62, 420–444 (2019).

    CAS  PubMed  Google Scholar 

Download references

Acknowledgements

The authors appreciate helpful comments from S. Jaensch, S. Singh, J. Caicedo, N. Rindtorff and all members of the Carpenter laboratory. The authors acknowledge funding support for S.N.C. and A.E.C. from the US National Institutes of Health (R35 GM122547 to A.E.C.).

Author information

Authors and Affiliations

  1. Imaging Platform, Broad Institute of MIT and Harvard, Cambridge, MA, USA

    Srinivas Niranj Chandrasekaran & Anne E. Carpenter

  2. Discovery Data Sciences, Janssen Pharmaceutica NV, Beerse, Belgium

    Hugo Ceulemans

  3. High Content Imaging Technology Center, Internal Medicine Research Unit, Pfizer Inc., Cambridge, MA, USA

    Justin D. Boyd

Authors
  1. Srinivas Niranj Chandrasekaran
    View author publications

    Search author on:PubMed Google Scholar

  2. Hugo Ceulemans
    View author publications

    Search author on:PubMed Google Scholar

  3. Justin D. Boyd
    View author publications

    Search author on:PubMed Google Scholar

  4. Anne E. Carpenter
    View author publications

    Search author on:PubMed Google Scholar

Contributions

The authors contributed equally to all aspects of the article.

Corresponding author

Correspondence to Anne E. Carpenter.

Ethics declarations

Competing interests

H.C. and J.D.B. are employed by Janssen and Pfizer, respectively. A.E.C. is on the Scientific and Technical Advisory Board of, has optional ownership interest in and receives income from Recursion. S.N.C. declares no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Glossary

Proteomic profiling

Measuring the levels of a large number of proteins in a sample, sometimes including their post-translationally modified forms.

Metabolomic profiling

Measuring the levels of a large number of metabolites in a sample.

Mechanism of action

(MOA). The description of how a compound interacts with a target and affects a biological system.

Side information

Further available measurements or metadata about samples that indirectly improve predictive performance.

Labels

Values for particular parameters in a given set of samples. For example, each compound in a dataset might have a mechanism of action label or a toxicity label.

Lead optimization

The process of narrowing down compounds after hit expansion to those with desired activity.

Brightfield images

Images captured from a sample without using any fluorescent illumination light.

Supervision

In machine learning, supervised learning aims for the system to predict the correct answers for each input, on the basis of examples. By contrast, in unsupervised learning the goal is to learn useful representations of each sample such that the similarities and differences among them can be observed.

Polypharmacology

The property of a compound whereby it interacts with more than a single target.

Neural network

A machine-learning architecture whereby features of a sample (for example, image pixels or image-derived metrics) are fed into a network of nodes, which collectively learn to produce the correct answer for that sample by each node adjusting its contribution (weight) to the final answer.

Hit expansion

The selection of compounds that were not tested in the primary screen, to broaden the diversity of the chemical space for hit selection. Compounds are selected on the basis of similarities in structure or biological activity to candidate hits.

SAR studies

An iterative process for lead optimization in which assays are applied to determine the effect of successive structural modifications to a compound on activity.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chandrasekaran, S.N., Ceulemans, H., Boyd, J.D. et al. Image-based profiling for drug discovery: due for a machine-learning upgrade?. Nat Rev Drug Discov 20, 145–159 (2021). https://doi.org/10.1038/s41573-020-00117-w

Download citation

  • Accepted:

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1038/s41573-020-00117-w

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing