Introduction

Microscopic detection and enumeration of microbes in environmental samples have provided fundamental information on the distribution of microbial life and global population estimates. The marine water is estimated to contain 1.2 × 1029 microbial cells1, whereas the subseafloor biomass is known to exceed this value, with estimates ranging from 2.9 to 5.4 × 1029 cells2,3,4. Aquatic samples are generally considered straightforward for detecting and enumerating microbial cells because the environmental matrix is water, which makes it easy to separate or trap particulate materials. In contrast, the abundance of particles in sediment samples significantly hinders microbial detection. Despite the vast size of the total subseafloor microbial population, volumetric calculations suggest that microbial cells occupy only a minor fraction (< 0.01%) of the entire sediment volume, even at the seafloor, where the highest microbial densities are typically observed5.

Additional challenges in detecting microbes in subseafloor sediment samples arise from the size and nature of sedimentary particles. The majority (> 90%) of marine sediments are composed of fine particles ranging from less than one micrometer to several tens of micrometers (Morono, 20236, and references therein7,8). These tiny particles are also known to adsorb dyes, such as DNA stains such as SYBR Green I (hereafter referred to as SYBR-I)9. The fluorescence intensity of SYBR-I is enhanced by more than a thousand-fold upon its association with DNA10. Although dye adsorption to the particle surface does not promote fluorescence enhancement, a high accumulation of dyes on the particles can produce a comparable fluorescence intensity independent of DNA association9. These particles pose a problem for microbial detection, but it has been discovered that they emit different fluorescence colors when observed through appropriate optical filters. The SYBR-DiCE (Discriminative Cell Enumeration)9,11 scheme is a technique that uses fluorescence color to distinguish microbial cells from non-cellular particles. Although most sedimentary particles can be recognized by their yellowish fluorescence, some particles also exhibit fluorescent colors similar to those of microbes. Therefore, extensive personnel training has been necessary to achieve reliable detection of microbial cells in marine sediment samples.

To achieve objective and reproducible detection of sedimentary microbes, we employed machine learning techniques to assist in particle identification. Deep learning, a class of machine learning methods using multilayered artificial neural networks, enables automated analysis of signals or data12,13. This technique is widely used in image processing across various fields and has recently been applied to the field of microscopic imaging. These advancements have improved the resolution, field of view, and depth of view of optical microscopy images14, enabled the prediction of subcellular protein localization15, and facilitated the classification of colorectal cancer from gland images16. In recent years, YOLO has been successfully applied to microscopic images, enabling the detection and classification of microorganisms such as yeast and those found in activated sludge17,18,19. However, applications have thus far been limited to images of large cells with higher pixel resolution and have not yet been extended to the recognition of tiny microbial cells with lower pixel resolution. Furthermore, this approach has not yet been applied to the recognition of microbial cells in sample images containing complex backgrounds such as mud and mineral particles.

In addition to YOLO‑based methods, several universal segmentation frameworks such as Mask R‑CNN and U‑Net (and related approaches) have demonstrated strong performance in biomedical and cultured‑cell imaging20,21,22. However, these models typically rely on clear cell boundaries and high pixel resolution—conditions rarely met in subseafloor sediment samples, where microbial cells are only 1–2 μm and surrounded by mineral grains that produce pseudo‑fluorescent backgrounds. Consequently, direct application of these methods generally requires extensive retraining and domain‑specific annotation and may not perform reliably in particle‑rich, low‑signal environments. Our approach specifically targets these challenges by combining SYBR‑DiCE–based, color‑adaptive particle detection (adaptive green‑intensity thresholds and the green‑to‑red ratio) with a compact ResNet‑based classifier optimized for the coarse‑pixel regime of microbial images.

In this study, we developed and validated a two‑step workflow to distinguish microbial cells from non‑cellular particles: (i) whole‑image detection of SYBR‑DiCE green‑fluorescent particles using adaptive green‑intensity thresholds and the green‑to‑red ratio (a color‑adaptive pre‑screening), and (ii) classification of cropped candidates with a compact ResNet‑based model. This method reduces computational costs for both deep learning and cell detection. Using the trained classifier, we detected and predicted subseafloor microorganisms from uncharacterized photographic images. Our results demonstrate the program’s effectiveness in accurately identifying microbial cells in complex sediment samples, providing a valuable tool for environmental microbiology. This approach enhances the objectivity and reproducibility of microbial detection, opening new possibilities for studying microbial communities in previously challenging environments. On the basis of our results, we discuss the utility of this program for microbial cell recognition and its potential applicability to a broader range of environmental samples.

Materials and methods

Preparation of microscopic images

To obtain images of microbial cells for subsequent deep learning analysis, we conducted epifluorescence microscopy observations via fluorescence staining. Marine sediment core samples were collected during multiple drilling expeditions: Integrated Ocean Drilling Program (IODP) Expedition 329 in the South Pacific Gyre (Site U1365B [23° 51′ 2.16″ S, 165° 38′ 39.84″ W, water depth: 5694.7 m], U1366D [26° 3′ 5.04″ S, 156° 53′ 39.84″ W, water depth: 5126.1 m], U1369C [39° 18′ 36.36″ S, 139° 48′ 3.24″ W, water depth: 5276.9 m], U1370E [41° 51′ 6.84″ S, 139° 6′ 21.96″ W, water depth: 5074.2 m], U1371F [45° 57′ 51.12″ S, 163° 11′ 2.04″ W, water depth: 5297.0 m]), Expedition 346 in the Japan Sea (Site U1424 [40° 11′ 24.00″ N, 138° 13′ 54.12″ E, water depth: 2807.32 m]) and the East China Sea (Site U1428 [31° 40′ 39.00″ N, 129° 1′ 59.88″ E, water depth: 724 m]), International Ocean Discovery Program (IODP) Expedition 357 in the Atlantis Massif (Site M0074 [30° 9′ 52.20″ N, 42° 7′ 19.20″ W, water depth: 1550 m]), Expedition 385 in the Guaymas basin (Site U1545C [27° 38′ 14.64″ N, 111° 53′ 19.68″ W, water depth: 1594.96 m]), Chikyu Expedition 906 in the Kumano Mud-Volcano (Site C9004 [33° 40′ 32.88″ N, 136° 34′ 5.16″ E, water depth: 1900 m]), Chikyu-SCORE expedition 910 in the west off Cape Erimo (Site C9033A [41° 48′ 8.28″ N, 142° 21′ 27.72″ E, water depth: 1068.5 m] and C9033B [41° 48′ 8.64″ N, 142° 21′ 28.08″ E, water depth: 1069.0 m]), and Kaiyo KY-11-E6 in the offshore Shimokita Peninsula (Site C9001 [41° 10′ 35.40″ N, 142° 12′ 1.80″ E, water depth 1180 m]). The collection years are detailed in the expedition proceedings. We employed the membrane staining method described by Morono et al. (2009)9 and Mori et al. (2021)11, with modifications. In brief, a formalin-fixed sediment slurry (25–50 µL, including 10–20 µL of sediment) was diluted with 375 µL of Tris-EDTA (TE) buffer (10 mM Tris-HCl and 1.0 mM EDTA, pH 8.0). Then, 50 µL of detergent mixture (100 mM EDTA, 100 mM sodium pyrophosphate, and 1% [v/v] Tween 80) and 50 µL of methanol were added. The sediment slurry was sonicated (Bioruptor UCD-250; Cosmo Bio, Tokyo, Japan) in an ice bath for 20 cycles of 30 s on at 200 W and 30 s off. The slurry was filtered through a black polycarbonate membrane (0.22 μm pore size, Isopore GTBP02500; Millipore). After air-drying, a quarter of the membrane was stained with 40 µL of SYBR-I solution (1/40 [v/v] SYBR-I [Thermo Fisher Scientific] in TE buffer). The stained filters were mounted on glass slides with a 1:2 mixture of VECTA shield and TE buffer and then sealed with cover glass. The samples were observed via an epifluorescence microscope (BX51, Olympus, Tokyo, Japan) with a 40× objective lens (UPLXAPO40×/0.95, ∞/0.11–0.23/FN26.5, Olympus, Tokyo, Japan). Multiple fluorescence microscopy images were acquired for each field of view using Olympus DP-72 and DP-74 digital cameras (Tokyo, Japan) with a 0.63× C-mount camera adapter (U-TV0.63XC, Olympus, Tokyo, Japan). Each exposure lasted 30 milliseconds, and images were saved as TIFF files. Excitation light intensity was set at 5, 10, or 20 to capture each photograph at varying background brightness levels.

Training data creation

Training data for deep learning were prepared by cropping the fluorescence microscopic images to reduce computational costs (The annotation program was published on Zenodo [https://doi.org/10.5281/zenodo.15193527]). Regions containing green fluorescent particles were automatically cropped on the basis of RGB color information, with the program targeting objects exhibiting green fluorescence. For detection, a “green threshold” and “green/red RGB” values were used. The green threshold, empirically determined from multiple source images, was set adaptively to 93% of the cumulative total green channel value (G value) of each image (Fig. 1a). In addition, a “green/red ratio” of 1.0 was applied to particles brighter than the green threshold. For particles with faint fluorescence (below the green threshold but a G value > 30), the “green/red ratio” was set to 1.5 (Fig. 1b-e). The detection logic was as follows:

$$\:\text{G-value}\:>30\:\:\:\cap\:\:\:\:\:\:\:\frac{\text{G-value}}{\text{R-value}}>1.0\:\:\:\:\:\:\:if\:\text{G-value}\:>\text{Green\:threshold}$$
(1)

・

$$\:\text{G-value}>30\:\:\:\:\cap\:\:\:\:\:\:\frac{\text{G-value}}{\text{R-value}}\:>1.5\:\:\:\:\:\:if\:\text{G-value}\:<\text{Green\:threshold}$$
(2)

・

Fig. 1
figure 1

(a) Determination of the green threshold from a cropped image, (b) conditions for detecting green fluorescent particles, (c–e) representative images of fluorescent particles (including microbial cells and non-cellular particles). Detection logic is defined in Eqs. (1) and (2).

Full size image

Morphological transformations (dilation) and local maximum detection in OpenCV, as well as local maximum detection in Scikit-Image, were used to suppress multiple detections of the same object. The identified green objects were cropped into 48 × 48-pixel squares centered on the object. Cropped images of 48 × 48 pixels were sufficient to encompass all green fluorescent particles. Each pixel corresponded to approximately 0.16 μm at the sample plane (based on the 40× objective with the 0.63× relay lens). Thus, a 48 × 48 pixel crop covered roughly 7.7 × 7.7 μm, fully encompassing single microbial cells (typically 1–2 μm) and small aggregates while minimizing redundant background. Cropped images were classified into either two classes (cells [P] and non-cellular particles [N]) or four classes (0: single cell, 1: multiple cells, 2: blurred/out-of-focus, 3: non-cellular particle) on the basis of visual inspection.

Deep learning

A classifier was developed to predict the presence of microbial cells via the prepared training data. The computational environment included Python 3.7.16, TensorFlow-GPU 2.9.0, Keras 2.9.0, CUDA 11.2, cuDNN 8.1, OpenCV 4.8.0, and Matplotlib 3.5.3 running in an Anaconda3 virtual environment with NVIDIA RTX A4000 (16 GB of memory). Adaptive moment estimation (ADAM)23 was used for optimization. The classified architecture was ResNet-18, a convolutional neural network with 18 layers and shortcut connections to reduce the computational cost24 (Fig. 2, S1) (Classification was performed with the Deep_learning_ResNet18 program, available on Zenodo [https://doi.org/10.5281/zenodo.15193527]).

Fig. 2
figure 2

Convolutional Neural Network (CNN) architecture of ResNet-18.

Full size image

The ResNet-18 model was chosen because of its capability to achieve stable and reliable class classification performance even when trained on small-scale datasets. The model accepts 48 × 48-pixel RGB images, extracting features via deep convolutional layers and residual blocks. This architecture includes an initial convolution layer (64 filters, 3 × 3), followed by residual blocks, spatial downsampling by convolution layers with stride 2, a global average pooling (GAP) layer, and a fully connected layer with Softmax activation.

Data augmentation was implemented to increase training data diversity, including rotations, reversals, and X/Y translations, as recommended by Shorten and Khoshgoftaar (2019)25. The “Image Data Generator” module (TensorFlow. keras. preprocessing. Image) was used with the following parameters: rotation = 180, width shift = 0.5, height shift = 0.5, horizontal flip = True, vertical flip = True, and random erasing26. Min–max normalization of the RGB channels was performed on the cropped image data by dividing each pixel value by 255. The cropped images were split into training and test datasets, with 20% of the training data reserved for validation. Training was repeated for up to 300 epochs, with early stopping on the basis of validation accuracy. The cross-entropy loss function was used:

$$\:Closs\:entropy\:loss\:=\:-\sum\:_{x}anotation\:class\:\left(x\right)\:\text{l}\text{n}\left(predicton\:class\:\right(x\left)\right)$$
(3)

The learning rate was initially set to 0.001, then reduced to 0.0005 after 75 training epochs and to 0.0001 after 150 epochs. We constructed both a two-class and a four-class classifier. Additionally, following pre-screening of Class 2 (blurred images) using the implemented program, we trained both two-class and three-class classifiers. The classifier performance was evaluated via reserved test data.

Detection and classification

The trained classifier and prediction program were applied to classify unknown microscopic images (The detect_and_prediction program available on Zenodo [https://doi.org/10.5281/zenodo.15193527] was employed to classify unknown microscopic images.). The same algorithms for green particle detection and image cropping were used. Cropped images containing green fluorescent particles were classified into the class with the highest probability predicted by the trained classifier. The performance metrics, including accuracy, recall, precision, and F-measure, were calculated as follows27:

$$\:Accuracy=\frac{(True\:positive\:+\:True\:Negative)}{(True\:Positive\:+\:\text{F}\text{a}\text{l}\text{s}\text{e}\:\text{P}\text{o}\text{s}\text{i}\text{t}\text{i}\text{v}\text{e}\:+\:\text{T}\text{r}\text{u}\text{e}\:\text{N}\text{e}\text{g}\text{a}\text{t}\text{i}\text{v}\text{e}\:+\:\text{F}\text{a}\text{l}\text{s}\text{e}\:\text{N}\text{e}\text{g}\text{a}\text{t}\text{i}\text{v}\text{e})}$$
(4)
$$\:Recall=\:\frac{True\:Positive}{(True\:Positive\:+\:False\:Negative)\:}$$
(5)
$$\:Precision=\:\frac{True\:Positive}{(True\:Positive\:+\:False\:Positive)}$$
(6)
$$\:F-measure=\:\frac{(2\:\times\:\:Precision\:\times\:\:Recall)}{(Precision\:+\:Recall)}$$
(7)

Recall measures the proportion of correctly predicted positives among all actual positives. Precision evaluates the proportion of correct predictions among all the predicted positives. The F-measure combines precision and recall. Values closer to 1 indicate better performance.

Results and discussion

Color-based pre-screening to reduce the computational cost in cell detection

We stained the sediment samples with SYBR-I according to the SYBR-DiCE method described by Morono et al. (2009)9. In this approach, a high concentration of SYBR-I dye (250 times the manufacturer’s recommendation) induces a color shift: microbial cells fluoresce green, whereas most non-cellular particles appear yellow. However, some non-cellular particles still exhibited green fluorescence. Our objective was to develop a trained classifier capable of accurately distinguishing green fluorescent particles, thus enabling reliable microbial cell detection.

Initially, we aimed to detect all green fluorescent particles for subsequent cropping and deep learning. Although SYBR I-stained microbial cells typically fluoresce green, some cultivated cells presented bright yellowish-green fluorescence, presumably due to excessive dye uptake (Fig. 1b). Simple thresholding on the basis of the green-to-red ratio (green/red > 1.0) also identified many non-cellular, dark yellowish particles. To address this, we implemented a “green threshold” to adaptively apply different green-to-red intensity ratios for green particle detection (Fig. 1a; see Materials and Methods for details). Additionally, we incorporated processes to suppress redundant detections and ensure a single detection of closely situated particles. As a result, all the green fluorescent particles were detected without redundancy. Each detected particle was cropped into a 48 × 48-pixel region and annotated for classification (Fig. S2). The percentage of overlapping green fluorescent objects was approximately 0.85% among all cropped images (137 out of 16,063). We minimized redundancy by employing morphological expansion and local maximum detection. However, the detection of images containing multiple cells affected the accuracy of microbial cell quantification. To address this, we subdivided Class P into two classes to separately quantify single cells and clusters of multiple cells. Furthermore, within Class N, we identified images where blurring hindered discrimination between cells and non-cellular particles; thus, we subdivided Class N into two classes as well. This resulted in the implementation of a four-class classification. We established the blur criterion by plotting histograms of the sharpness metric, calculated as the Laplacian variance—a standard focus measure that quantifies the dispersion of pixel intensity gradients—for Class 2 images, with the sharpness values on the X-axis and frequency on the Y-axis (Fig. S3). As the majority of cases had sharpness values close to zero, this value was chosen as the threshold for designating Class 2. The result, we established both two-class (cells and non-cellular particles) and four-class (single cell, multiple cells, blurred/out-of-focus, and non-cellular particles) classification schemes, annotating a total of 16,063 green fluorescent objects and assigning 9,733 as cells (class P) and 6,330 as non-cellular particles (class N) in the two-class scheme and 7,821 as single cells (class 0), 1,912 as multiple cells (class 1), 2,363 as blurred (class 2), and 3,967 as non-cellular particles (class 3) in the four-class scheme (Fig. 3 shows examples of each class).

Fig. 3
figure 3

Cropped images and their classification. Two-class (class P: cells. class N: non-cellular particles) and Four-class (class 0: single microbial cell. class 1: multiple microbial cells. class 2: blurred (out-of-focus). class 3: non-cellular particles).

Full size image

Classifier accuracy and prediction of unknown images

Using two-class annotation, the dataset was split into training (cells: 7,532; non-cellular particles: 5,535) and test data (cells: 2,201; non-cellular particles: 795), with 20% of the training data reserved for validation. Training data were shuffled prior to each session to prevent bias. We evaluated the accuracy of the trained classifier in detecting and classifying microbial cells from unknown microscopic images. The trained classifier achieved 94.1% accuracy on annotated test data (Fig. 4).

Fig. 4
figure 4

Training curves of loss (a) and accuracy (b) for the two-class model.

Full size image

We further evaluated classifier performance by comparing predictions to human annotations via a confidence index histogram (Fig. 5). This figure depicts outcomes derived from training the two-class classifier. The classifier performed better on cell images (2,155/2,201, 98.0% correct) than on non-cellular particle images (665/795, 83.6% correct). Most misclassifications among cells involved dark fluorescence (34/46, Fig. 5a, c), whereas misclassifications among particles were associated with dark or blurred images (73/130 or 36/130, respectively) (Fig. 5a, d). Some incorrect classifications for non-cellular particles occurred at high confidence levels, indicating that dark and blurred images critically affected accuracy (Fig. S4).

Fig. 5
figure 5

(a) Confusion matrix comparing human annotation and predictions by the two-class classifier. Red indicates false negatives; blue indicates false positives. (b–d) Examples of misclassified images, (e,f) histograms of each class.

Full size image

To improve classification, we subdivided the classes further: cells into single and multiple cells and non-cellular particles into blurred and non-cellular particles. To compensate for smaller datasets due to subdivision, we augmented the image data with rotations, shifts, flips, and random erasing, following Shorten and Khoshgoftaar (2019)25. The four-class classifier trained with these augmented data achieved an accuracy of 88.8% (Fig. 6). Metrics for recall, precision, and F-measure (Table 1) revealed that the four-class classifier performed comparably to the two-class classifier, except for the blurred image class, which had lower metrics.

Fig. 6
figure 6

Training curves of loss (a) and accuracy (b) for the four-class model.

Full size image
Table 1 Calculation results of the metrics for each class in the trained classifier.
Full size table

Detailed analysis and optimization

To gain further insights, we examined the classification results of the four-class classifier in detail, as shown in Fig. 7. The average processing time was 7.88 s per image, corresponding to approximately 3–4 min per sample (comprising 20–30 fields of view). For the four-class classification, misclassifications between subclasses within the same biological group (Class 0 ↔ Class 1 or Class 2 ↔ Class 3) were treated as within-category errors, rather than false positives or negatives. Misclassifications of non-cell categories as cell (Class 2/3 → Class 0/1) and of cell categories as non-cell (Class 0/1 → Class 2/3) were regarded as false positives and false negatives, respectively, as these constitute the most critical sources of error. A detailed examination of the four-class classifier revealed that combining single- and multiple-cell classes minimized false negatives (1.2% of the total, Fig. 7a). The learning model tended to misclassify images with low green intensity, even under the four-class categorization (Fig. S5). This finding indicates that dark-fluorescence images are particularly prone to misclassification, confirming that fluorescence intensity critically affects prediction accuracy. Most false negatives had a confidence index less than 0.7 (covering 21/27 false negatives), suggesting that applying a cutoff could improve accuracy. Among the blurred images, 63 out of 138 were misclassified as cells, and a large fraction of these (50 out of 69 false negatives) occurred even at high confidence levels, indicating that blurred images themselves strongly induce classification errors and should therefore be minimized during image acquisition. For blurred images, many false positives occurred at high confidence levels, likely due to variability in the degree of blurring. Solutions include improving image acquisition to avoid blurring, applying image deblurring algorithms28, or manually pre-screening blurred images. The false positives in the non-cellular particle class were due mainly to faint fluorescence, which is challenging to annotate. However, these cases were rare after a confidence index cutoff of 0.7 (1.71% of the total) was applied. By applying a 0.7 confidence index cutoff and pre-screening blurred images, the accuracy improved to 93.7% for the four-class classifier and 96.6% for the two-class classifier (Fig. S5, Fig.S6 and Table S1). The YOLO model has been shown to achieve up to 93.7% accuracy in classifying microorganisms in microscope images18. However, these studies typically utilized images captured at high magnifications, containing minimal non-biological particles such as debris or sediment. Therefore, our model can be regarded as highly accurate for cell classification at low magnification, even in challenging environments with abundant non-cellular particles.

Fig. 7
figure 7

(a) Confusion matrix comparing human annotation and predictions by the four-class classifier. Red is false negatives; blue is false positives; green indicates within-category errors (Class 0 ↔ 1 or Class 2 ↔ 3). Numbers in parentheses show cases with confidence index > 0.7. (b–f), Examples of misclassified images, (g–j) histograms for each class.

Full size image

Deep‑learning‑based frameworks continue to advance rapidly, but most universal cell‑detection models remain constrained by imaging quality and sample complexity22. In contrast to Mask R‑CNN, U‑Net, and related architectures—which perform best for biomedical or tissue imaging with well‑defined morphology—our model addresses a different regime: low‑magnification, particle‑rich subseafloor sediments characterized by low signal‑to‑noise ratios. This application‑specific design emphasizes that domain adaptation is often essential for microbial detection in complex environmental matrices, where universal models often do not yield reliable quantification without extensive retraining or preprocessing20,21.

Future implementation and applications

Our trained classifier can be implemented in two ways: as a fully automated cell detection system and as an aid for human identification. The highest accuracy was achieved by merging single and multiple cell classes and excluding blurred images at the input stage. For automated detection, an algorithm for counting cells per cropped image would improve quantification. For human-assisted identification, the four-class classifier already provides sufficient accuracy when a confidence index cutoff of 0.7 is used. This system enables the reliable identification of microbial cells in complex sediment samples without requiring extensive expert training.

Conclusion

We developed a novel deep learning-based classifier for coarse-pixel microbial image classification. The system successfully classified green fluorescent particles from microscopic images, achieving 94.1% accuracy in the two-class scheme and 88.8% accuracy in the four-class scheme. Misclassifications were most common below a 0.7 confidence index and were influenced by blurred images. By applying a 0.7 cutoff and pre-screening blurred images, the accuracy improved to 93.6% (four-class) and 96.6% (two-class). Most false negatives and false positives were eliminated, and the remaining misclassifications were minimal. This approach allows accurate microbial cell recognition without specialized training, facilitating broader use in environmental microbiology.