Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Structured illumination microscopy with noise-controlled image reconstructions

Abstract

Super-resolution structured illumination microscopy (SIM) has become a widely used method for biological imaging. Standard reconstruction algorithms, however, are prone to generate noise-specific artifacts that limit their applicability for lower signal-to-noise data. Here we present a physically realistic noise model that explains the structured noise artifact, which we then use to motivate new complementary reconstruction approaches. True-Wiener-filtered SIM optimizes contrast given the available signal-to-noise ratio, and flat-noise SIM fully overcomes the structured noise artifact while maintaining resolving power. Both methods eliminate ad hoc user-adjustable reconstruction parameters in favor of physical parameters, enhancing objectivity. The new reconstructions point to a trade-off between contrast and a natural noise appearance. This trade-off can be partly overcome by further notch filtering but at the expense of a decrease in signal-to-noise ratio. The benefits of the proposed approaches are demonstrated on focal adhesion and tubulin samples in two and three dimensions, and on nanofabricated fluorescent test patterns.

This is a preview of subscription content, access via your institution

Access options

Buy this article

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Noise propagation in SIM reconstructions.
Fig. 2: Noise-controlled SIM reconstructions.
Fig. 3: Trade-off between noise and contrast in SIM reconstructions.
Fig. 4: Noise-controlled 3D-SIM reconstructions.
Fig. 5: Resolution improvement and noise enhancement in deconvolution of GFP-zyxin dataset.

Similar content being viewed by others

Data availability

Data are available at https://doi.org/10.4121/12942932.

Code availability

MATLAB code is available at https://github.com/qnano/simnoise. ImageJ code for 2D-SIM is available at https://github.com/fairSIM.

References

  1. Neil, M. A. A., Juskaitis, R. & Wilson, T. Method of obtaining optical sectioning by using structured light in a conventional microscope. Opt. Lett. 22, 1905–1907 (1997).

    Article  CAS  PubMed  Google Scholar 

  2. Heintzmann, R. & Cremer, C. Laterally modulated excitation microscopy: improvement of resolution by using a diffraction grating. Proc. SPIE 3568, 185–196 (1999).

    Article  Google Scholar 

  3. Gustafsson, M. G. L. Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy. J. Microsc. 198, 82–87 (2000).

    Article  CAS  PubMed  Google Scholar 

  4. Gustafsson, M. G. L. et al. Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination. Biophys. J. 94, 4957–4970 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Schermelleh, L. et al. Subdiffraction multicolor imaging of the nuclear periphery with 3D structured illumination microscopy. Science 320, 1332–1336 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Heintzmann, R. & Huser, T. Super-resolution structured illumination microscopy. Chem. Rev. 117, 13890–13908 (2017).

    Article  CAS  PubMed  Google Scholar 

  7. Kner, P., Chhun, B. B., Griffis, E. R., Winoto, L. & Gustafsson, M. G. L. Super-resolution video microscopy of live cells by structured illumination. Nat. Methods 6, 339–342 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Shao, L., Kner, P., Rego, E. H. & Gustafsson, M. G. L. Super-resolution 3D microscopy of live whole cells using structured illumination. Nat. Methods 12, 1044–1046 (2011).

    Article  CAS  Google Scholar 

  9. Fiolka, R., Shao, L., Rego, E. H., Davidson, M. W. & Gustafsson, M. G. L. Time-lapse two-color 3D imaging of live cells with doubled resolution using structured illumination. Proc. Natl Acad. Sci. USA 109, 5311–5315 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Heintzmann, R., Jovin, T. & Cremer, C. Saturated patterned excitation microscopy – a concept for optical resolution improvement. J. Opt. Soc. Am. B 19, 1599–1609 (2002).

    Article  Google Scholar 

  11. Gustafsson, M. G. L. Nonlinear structured-illumination microscopy: wide-field fluorescence imaging with theoretically unlimited resolution. Proc. Natl Acad. Sci. USA 102, 13081–13086 (2005).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Rego, E. H. et al. Nonlinear structured-illumination microscopy with a photoswitchable protein reveals cellular structures at 50-nm resolution. Proc. Natl Acad. Sci. USA 109, E135–E143 (2012).

    Article  CAS  PubMed  Google Scholar 

  13. Li, D. et al. Extended-resolution structured illumination imaging of endocytic and cytoskeletal dynamics. Science 349, aab3500 (2015).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  14. Wicker, K., Mandula, O., Best, G., Fiolka, R. & Heintzmann, R. Phase optimization for structured illumination microscopy. Opt. Express 21, 2032–2049 (2013).

    Article  PubMed  Google Scholar 

  15. Křížek, P., Lukeš, T., Ovesný, M., Fliegel, K. & Hagen, G. M. SIMToolbox: a MATLAB toolbox for structured illumination fluorescence microscopy. Bioinformatics 32, 318–320 (2016).

    PubMed  Google Scholar 

  16. Müller, M., Mönkemöller, V., Hennig, S., Hübner, W. & Huser, T. Open-source image reconstruction of super-resolution structured illumination microscopy data in ImageJ. Nat. Commun. 7, 10980 (2016).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  17. Ball, G. et al. SIMcheck: a toolbox for successful super-resolution SIM imaging. Sci. Rep. 5, 15915 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Demmerle, J. et al. Strategic and practical guidelines for successful structured illumination microscopy. Nat. Protoc. 12, 988–1010 (2017).

    Article  CAS  PubMed  Google Scholar 

  19. Sahl, S. J. et al. Comment on extended-resolution structured illumination imaging of endocytic and cytoskeletal dynamics. Science 352, 527 (2016).

    Article  CAS  PubMed  Google Scholar 

  20. Li, D. et al. Response to comment on ‘Extended-resolution structured illumination imaging of endocytic and cytoskeletal dynamics’. Science 352, 527 (2016).

    CAS  PubMed  Google Scholar 

  21. Righolt, C. H. et al. Image filtering in structured illumination microscopy using the Lukosz Bound. Opt. Express 21, 24431–24451 (2013).

    Article  PubMed  Google Scholar 

  22. Fried, D. L. Noise in photo-emission current. Appl. Opt. 4, 79–80 (1965).

    Article  Google Scholar 

  23. Hu, S. et al. Structured illumination microscopy reveals focal adhesions are composed of linear subunits. Cytoskeleton 72, 235–245 (2015).

    Article  PubMed  CAS  Google Scholar 

  24. Unser, M., Trus, B. L. & Steven, A. C. A new resolution criterion based on spectral signal-to-noise ratio. Ultramicros 23, 39–52 (1987).

    Article  CAS  Google Scholar 

  25. Nieuwenhuizen, R. P. J. et al. Measuring image resolution in optical nanoscopy. Nat. Methods 10, 557–562 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Chakrova, N., Heintzmann, R., Rieger, B. & Stallinga, S. Studying different illumination patterns for resolution improvement in fluorescence microscopy. Opt. Express 23, 31367–31383 (2015).

    Article  PubMed  Google Scholar 

  27. Heintzmann, R. Estimating missing information by maximum likelihood deconvolution. Micron 38, 136–144 (2007).

    Article  PubMed  Google Scholar 

  28. Perez, V., Chang, B.-J. & Stelzer, E. H. K. Optimal 2D-SIM reconstruction by two filtering steps with Richardson–Lucy deconvolution. Sci. Rep. 6, 37149 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Wang, H. et al. Deep learning enables cross-modality super-resolution in fluorescence microscopy. Nat. Methods 16, 103–110 (2019).

    Article  CAS  PubMed  Google Scholar 

  30. Hoffman, D. P., Slavitt, I. & Fitzpatrick, C. A. The promise and peril of deep learning in microscopy. Nat. Methods 18, 131–132 (2021).

    Article  CAS  PubMed  Google Scholar 

  31. Huang, X. et al. Fast, long-term, super-resolution imaging with Hessian structured illumination microscopy. Nat. Biotechnol. 36, 451–459 (2018).

    Article  CAS  PubMed  Google Scholar 

  32. Markwirth, A. et al. Video-rate multi-color structured illumination microscopy with simultaneous real-time reconstruction. Nat. Commun. 10, 4315 (2019).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  33. Ströhl, F. & Kaminski, C. F. Speed limits of structured illumination microscopy. Opt. Lett. 42, 2511–2514 (2017).

    Article  PubMed  Google Scholar 

  34. Chen, B.-C. et al. Lattice light-sheet microscopy: Imaging molecules to embryos at high spatiotemporal resolution. Science 346, 1257998 (2014).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  35. van der Horst, J., Trull, A. K. & Kalkman, J. Deep-tissue label-free quantitative optical tomography. Optica 7, 1682–1689 (2020).

    Article  Google Scholar 

  36. Boulanger, J., Pustelnik, N., Condat, L., Sengmanivong, L. & Piolot, T. Nonsmooth convex optimization for structured illumination microscopy image reconstruction. Inverse Prob. 34, 095004 (2018).

    Article  Google Scholar 

  37. Weigert, M. et al. Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat. Methods 15, 1090–1097 (2018).

    Article  CAS  PubMed  Google Scholar 

  38. Jin, L. et al. Deep learning enables structured illumination microscopy with low light levels and enhanced speed. Nat. Commun. 11, 1934 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Krull, A. et al. Noise2Void—learning denoising from single noisy images. in Proc. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2124–2132 (IEEE, 2019).

  40. Reinhard, M. et al. An alpha-actinin binding site of zyxin is essential for subcellular zyxin localization and alpha-actinin recruitment. J. Biol. Chem. 274, 13410–13418 (1999).

    Article  CAS  PubMed  Google Scholar 

  41. Suresh Babu, S. et al. Mechanism of stretch-induced activation of the mechanotransducer zyxin in vascular cells. Sci. Signal. 5, ra91 (2012).

    Article  PubMed  CAS  Google Scholar 

  42. Yoshigi, M., Hoffman, L. M., Jensen, C. C., Yost, H. J. & Beckerle, M. C. Mechanical force mobilizes zyxin from focal adhesions to actin filaments and regulates cytoskeletal reinforcement. J. Cell Biol. 171, 209–215 (2005).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  43. Schlapak, R. et al. Painting with biomolecules at the nanoscale: biofunctionalization with tunable surface densities. Nano Lett. 12, 1983–1989 (2012).

    Article  CAS  PubMed  Google Scholar 

  44. Enguita-Marruedo, A. et al. Live cell analyses of synaptonemal complex dynamics and chromosome movements in cultured mouse testis tubules and embryonic ovaries. Chromosoma 127, 341–359 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  45. Peters, A. H., Plug, A. W., van Vugt, M. J. & de Boer, P. A drying-down technique for the spreading of mammalian meiocytes from the male and female germline. Chromosome Res. 5, 66–68 (1997).

    Article  CAS  PubMed  Google Scholar 

  46. Schücker, K., Holm, T., Franke, C., Sauer, M. & Benavente, R. Elucidation of synaptonemal complex organization by super-resolution imaging with isotropic resolution. Proc. Natl Acad. Sci. USA 112, 2029–2033 (2015).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  47. Heintzmann, R. et al. Calibrating photon counts from a single image. Preprint at https://arxiv.org/abs/1611.056541611.05654 (2016).

  48. Bakx, J. L. Efficient computation of optical disk readout by use of the chirp z transform. Appl. Opt. 41, 4897–4903 (2002).

    Article  PubMed  Google Scholar 

  49. Wicker, K. Non-iterative determination of pattern phase in structured illumination microscopy using autocorrelations in Fourier space. Opt. Express 21, 24692–24701 (2013).

    Article  PubMed  Google Scholar 

  50. Stallinga & Rieger, B. Accuracy of the Gaussian Point Spread Function model in 2D localization microscopy. Opt. Express 18, 24461–24476 (2010).

    Article  CAS  PubMed  Google Scholar 

  51. Ingaramo, M. et al. Richardson–Lucy deconvolution as a general tool for combining images with complementary strengths. ChemPhysChem 15, 794–800 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  52. Ströhl, F. & Kaminski, C. F. A joint Richardson–Lucy deconvolution algorithm for the reconstruction of multifocal structured illumination microscopy data. Methods Appl. Fluoresc. 3, 014002 (2015).

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We thank M. Booth and B. Rieger for stimulating research advice, A. York for suggesting the binomial random splitting of Poisson-distributed variables and W. Baarends for kindly providing mCherry-SYCP3 samples. C.S. was supported by a Junior Research Fellowship through Merton College (Oxford, UK). L.S. acknowledges support by the Wellcome Trust Strategic Award 107457 and the European Research Council MSC ITN grant no. 766181. N.C. acknowledges European Research Council grant no. 648580. C.H. acknowledges support from the Netherlands Organization for Scientific Research (ZonMW-435002021). S.H. acknowledges support by NanoNextNL, a consortium of the Dutch government and 130 public and private partners.

Author information

Authors and Affiliations

Authors

Contributions

Imaging experiments were done by C.S.S., J.A.S., L.S., N.C., W.v.C. and A.B.H. J.A.S., S.H., Y.V., C.W.H. and J.P.H. designed and manufactured nanofabricated test samples. C.S.S., N.C., M.M. and S.S. analyzed data. S.S. derived theory, wrote the paper and supervised the research. All authors read and approved the manuscript.

Corresponding author

Correspondence to Sjoerd Stallinga.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Additional information

Peer review Information Nature Methods thanks Florian Ströhl and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Rita Strack was the primary editor on this article and managed its editorial process and peer review in collaboration with the rest of the editorial team.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Noise-controlled SIM reconstructions of GFP-zyxin protein in focal adhesions (green) and noise fraction map (magenta) over full FOV.

a–d, State-of-art SIM (w = 5 × 10−4), true-Wiener SIM, flat-noise SIM, and notch-filtered SIM reconstructions. Contours of the noise fraction map are added in white with contour level indicated. In all reconstructions the noise fraction is lowest in the foreground features and highest in the background region outside the cell. Overall, flat-noise SIM and true-Wiener SIM offer the lowest, and notch-filtered SIM the highest noise enhancement. Scale bar 5 μm.

Extended Data Fig. 2 Multi-color noise-controlled 2D-SIM reconstructions.

a, Combined widefield, true-Wiener SIM, flat-noise SIM, and notch-filtered SIM reconstructions of a fluorescent test slide of a bovine pulmonary artery endothelial cell (red channel: mitochondria labeled with MitoTracker Red, green channel: actin labeled with Alexa Fluor 488, blue channel: DNA labeled with DAPI). Note that due to embedding in hardening mounting medium, cells are flattened and 3D nuclear morphology is compromised. b-f, Insets of the DAPI channel comparing state-of-the-art SIM with clear noise amplification artifact to the noise-controlled SIM reconstructions. The SSNR in the DAPI channel is low in this example case, due to reduced signal intensity and compromised morphology. The low SSNR is properly taken into account by the noise-controlled SIM reconstructions, without introducing artifacts, but not by the state-of-the-art SIM reconstruction. Scale bar (a) 10 μm, scale bar (b-f) 5 μm.

Extended Data Fig. 3 Noise propagation in DMD-SIM.

a, Reconstructions of Alexa Fluor 488 labeled actin filaments in a bovine pulmonary artery endothelial cell with the iterative pattern-illuminated Fourier Ptychography (piFP) algorithm (see Supplementary Note 1) and with a band-pass regularization approach for flat-noise SIM. b, Comparison of flat-noise SIM to a widefield reconstruction obtained by summing the whole set of acquired images. c-e, Insets of the boxed region in (a) and (b). Both piFP and flat-noise SIM offer a resolution improvement, but piFP has better contrast than flat-noise SIM. The piFP reconstruction shows corrugated line structures and punctuated features (upper right of insets), similar to the structured noise artifact in state-of-the-art SIM with line illumination patterns, flat-noise SIM shows this to a lesser degree. Scale bar (a,b) 10 μm, scale bar (c-e) 4 μm.

Extended Data Fig. 4 Flat-noise SIM provides better visibility of high spatial frequency structures.

a–d, Widefield, true-Wiener, flat-noise, and notch-filtered SIM reconstructions of a nanofabricated test structure of lines with 140 nm pitch. The line pattern is just visible in flat-noise, and notch-filtered SIM but overshadowed by the noise pattern with uneven distribution of noise over spatial frequencies in true-Wiener SIM. Scale bar 1 μm.

Extended Data Fig. 5 Noise-controlled 2D-SIM of synaptonemal complex.

a–d, Widefield, and true-Wiener, flat-noise and notch-filtered SIM reconstructions of the mCherry-CSYCP3 protein in the synaptonemal complex. e–h, Line profiles along the lines indicated in (b). The SIM reconstructions reveal the two cable sub-structure with a line distance of around 200 nm, flat-noise SIM has less contrast but shows smoother lines and no background noise structure. Scale bar 3 μm.

Extended Data Fig. 6 Cross-sections in xy and yz-planes of 3D-reconstructions of tubulin (green) and noise fraction maps (magenta) for different camera exposure times.

a, Widefield, (b–d) state-of-the-art SIM for low, medium and high regularization, (e) true-Wiener SIM, (f) flat-noise SIM, and (g) notch-filtered SIM. The dashed lines in (a) indicate the location of the xz and xy slices. Scale bar 3 μm.

Extended Data Fig. 7 Widefield and noise-controlled 3D-SIM reconstructions of a 100 nm bead layer sample.

a,b, Widefield, (c,d) true-Wiener SIM, (e,f) flat-noise SIM, (g,h) notch-filtered SIM. The white box in (c) indicates the insets (b,d,f,h). i,j, SSNR of the SIM reconstructions without (i) and with (j) notch filtering. The data is averaged over rings in Fourier space and the plot is on a logarithmic scale according to log10(1+SSNR). The red line indicates the (ring averaged) support of the SIM-OTF, the white line indicates the SSNR = 5 region in Fourier space used for the extrapolation of the true-Wiener regularization filter. k, FRC curves for SIM obtained from 4 repeated acquisitions of the bead layer sample. The FRC resolution is 106.3 ± 0.5 nm, very close to the extended SIM diffraction limit 1/(2NA/λ + 2/p) = 99 nm for the estimated pattern pitch p = 416 nm, consistent with the relatively high signal level (peak pixel intensities above 104 detected photons) and the broad support of SSNR above one in spatial frequency space. Scale bar (a,c,e,g) 3 μm, scale bar (b,d,f,h) 1 μm.

Extended Data Fig. 8 Widefield and 3D noise-controlled SIM reconstructions of a bovine pulmonary artery endothelial cell.

a, Widefield, (b) true-Wiener SIM, (c) flat-noise SIM, (d) notch-filtered SIM (BPAEC, red channel: mitochondria labeled with Alexa Fluor 594, green channel: actin labeled with FITC, blue channel: DNA labeled with DAPI). Scale bar 5 μm.

Extended Data Fig. 9 Widefield and 3D noise-controlled SIM reconstructions of a mouse C127 cell.

a, Widefield, (b) true-Wiener SIM, (c) flat-noise SIM, (d) notch-filtered SIM (magenta channel: DNA labeled with DAPI, green channel: H3K4me3 labeled with Alexa Fluor 488, blue channel: DNA labeled with DAPI). Scale bar 5 μm.

Extended Data Fig. 10

Widefield, state-of-the-art SIM and noise-controlled 3D-SIM reconstructions for one timeframe of the 15 timeframe, 7-layer dataset of H2B-GFP histone in a live HeLa cell. Scale bar 6 μm.

Supplementary information

Supplementary Information

Supplementary Figs. 1–7 and Note.

Reporting Summary

Supplementary Video 1

Impact of regularization parameter on state-of-the-art SIM.

Supplementary Video 2

Noise-controlled SIM of 10× repeated acquisition of GFP-zyxin.

Supplementary Video 3

Noise controlled 3D SIM of tubulin, signal level 1.

Supplementary Video 4

Noise controlled 3D SIM of tubulin, signal level 2.

Supplementary Video 5

Noise controlled 3D SIM of tubulin, signal level 3.

Supplementary Video6

Noise controlled 3D-SIM of tubulin, signal level 4.

Supplementary Video 7

Noise controlled 3D three-color SIM of BPAEC cell.

Supplementary Video 8

Noise controlled 3D two-color SIM of C127 cell.

Supplementary Video 9

Noise controlled 3D-SIM of live HeLa cell.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Smith, C.S., Slotman, J.A., Schermelleh, L. et al. Structured illumination microscopy with noise-controlled image reconstructions. Nat Methods 18, 821–828 (2021). https://doi.org/10.1038/s41592-021-01167-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue date:

  • DOI: https://doi.org/10.1038/s41592-021-01167-7

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing