AgriSPEC: A smartphone-based, compact biospeckle imager for assessing seed viability

Abstract

Seeds are a vital cornerstone of agriculture and their capability to germinate is vital for combating global food insecurity by ensuring high crop yields. We introduce ‘AgriSPEC,’ a custom-built, portable imaging device that utilizes a smartphone and laser speckle imaging to assess seed viability by visualizing and analyzing biological activity levels within multiple samples simultaneously. Our prototype imager comprises a compact attachment that incorporates all components for optimal illumination of the samples. Additionally, we have developed a processing algorithm that combines RGB and speckle images to generate graphical maps representing seed biological activity. Our results demonstrate the algorithm’s ability to generate maps with enhanced contrast as compared to existing methods. We validate the effectiveness of our device by performing standard germination tests for seed viability. Our proposed low-cost, real-time activity visualizer allows stakeholders to integrate this technology into seed selection protocols thereby promoting sustainability in agriculture by optimizing crop inputs.

Introduction

Establishing a stable and abundant food supply is currently a paramount concern in light of the ever-growing global population, which is expected to touch nearly 10 billion by the year 2050, and the challenges associated with climate change. Over time, extensive research has been done for developing seeds that exhibit enhanced traits such as disease resistance, improved yield potential, and increased tolerance to environmental stressors. By carefully selecting superior seeds, farmers can significantly enhance their crop yields, thereby ensuring reliable and sufficient food production¹. The selection of superior seeds brings a multitude of advantages, chief amongst them being that improved seeds can help crops withstand harsh environmental conditions, such as drought, extreme temperatures, or pests, reducing crop losses and thereby safeguarding farmers’ livelihoods. Resilient seeds not only minimize the need for excessive pesticide and water usage, resulting in more efficient and environmentally friendly agricultural practices, but also aid farmers in increasing crop yields without expanding their farmland, thereby preserving precious natural resources and alleviating the pressure on ecosystems.

Numerous techniques have been explored for assessing the quality of plant propagules such as roots and agricultural seeds, which are based on visual analysis methods, an example being thermal imaging, which correlates observed temperature differences to overall quality^2,3. Despite being cost-effective, thermal imaging methods suffer from erratic thermal fingerprints in uncontrolled environments⁴). Hyper-spectral imaging techniques^5,6 have also been used for grading corn, maize, and tomato seeds with high accuracies but the cost of the hardware, coupled with considerable data storage and processing power requirements, have made it impractical for field applications⁷. In recent times, laser speckle imaging (LSI) has emerged as a powerful tool for monitoring plant physiology, detecting stress factors, and enhancing crop management practices. Speckle patterns refer to the fluctuating, grainy intensity patterns that are observed when coherent light (such as that from the laser) is incident on a rough surface. These distinctive patterns are an outcome of interference between scalar field components that scatter from the surface with random phases and amplitudes. Speckle imaging provides the capability to non-destructively visualize transient phenomena (such as biological activity) within the sample of interest via spatial and temporal statistics of the observed patterns. For plant systems in particular, speckle imaging has been successfully applied as an unobtrusive, non-destructive technique that encodes information regarding the underlying microcirculation and inherent metabolic activity of the sample under consideration. The speckle patterns generated when agri-seeds are illuminated with coherent light are observed to be transient in nature and this dynamic behavior is attributed to the Brownian motion of certain cell organelles and nutrient and water molecules. This observed ‘activity’ can, in turn, be robustly correlated with the germination potential of agri-seeds^8,9. A commonly used method to estimate the viability of seeds¹⁰ involves inducing germination in pre-soaked seeds under optimized temperature and moisture conditions. LSI has been used to distinguish between seeds of high and low quality based on observed activity levels¹¹ and therefore, this becomes a promising method for detecting the vigor and vitality of the sample without involving tedious experimental protocols to induce germination.

Despite its enormous potential for addressing multiple use cases in the context of agriculture, the adoption of biospeckle imaging technology has been impeded by the lack of suitable hardware and processing algorithms that are computationally fast, efficient, and have high throughputs. In this paper, we describe our design of a portable, smartphone based speckle imaging device, which we have named ‘AgriSPEC’, in a form that integrates all the necessary components required to illuminate and record speckle images thereby overcoming many of the complexities associated with field use. The feasibility of smartphones for laser biospeckle analyses as such has been described in earlier reports but their utility has been demonstrated as an image capture device only which required additional modules for illumination etc.¹². Other previously reported embodiments of portable speckle imagers have used relatively expensive components such as embedded GPU platforms and scientific cameras^13,14 although bringing in portability to the overall hardware footprint. We also introduce the speckle activity map (SAM) algorithm, which is a novel graphical technique for analyzing speckle images; this technique not only generates activity maps with superior contrast (described in the following sections) but also addresses the important issue of throughput wherein multiple samples can be imaged and processed simultaneously. Previous studies have proposed numerical techniques such as inertia moment (IM)^15,16, average value of differences (AVD)¹⁷, for estimating activity levels. However, these techniques cannot process multiple samples within a single frame in one go without additional clustering and segmentation steps. Graphical analysis methods have become increasingly popular in recent times as they offer a more intuitive view of the information about the sample. Existing graphical analysis methods, such as the generalized difference (GD), weighted generalized difference (WGD)¹⁸), and Fujii’s method¹⁹ are well-known approaches for analyzing speckle images. These methodologies generate color-coded activity maps that provide valuable information about the analyzed samples. We benchmark our graphical analysis technique and demonstrate its capability to outperform traditional methods by producing superior outputs.

The paper is organized as follows: section “Results” describes the protocol followed for preparing the seed samples, details of the AgriSPEC smartphone based device, and the data capture workflow. Section “Discussion” introduces the SAM algorithm and the steps therein. In section “Methods”, we discuss SAM performance using synthetic data, publicly available datasets as well as experimental data acquired by our AgriSPEC device.

Results

In order to benchmark the output of various biospeckle analysis algorithms, we followed Kulkarni et al.²⁰ and used synthetically generated, spatially and temporally speckle image sequences. Such ground truth data can be used to compare multiple algorithms to analyze contrast performances. We have taken a similar approach to compare the output of our technique with the Fujii and GD techniques. Speckle sequences comprising of N = 25 and N = 50 images generated synthetically as per the method described by Federico et al.²¹. Figure 1 presents the activity maps of synthetic data generated for various processing algorithms. As shown schematically in Fig. 2a, the generated activity maps have 5 distinct regions (R1–R5) with different levels of temporal activity with region R1 having the lowest activity and R5 with the highest. The activity maps computed for the N = 25 and N = 50 image sequences are depicted in Fig. 1 wherein plot labeled (a) and (b) represents the activity maps for Fujii’s method for N = 25 and N = 50, respectively. Similarly, plots labeled (c) and (d) depict the activity maps computed using generalized differences for N = 25 and N = 50, respectively. Finally, the activity maps generated by using our proposed SAM algorithm are presented in in Fig. 1e, f. The enhanced contrast across all five regions for the SAM outputs as compared to the other techniques (Fujii, GD) can be readily observed.

Fig. 1: Visual temporal activity maps calculated using synthetic data for Fujii’s and the Generalized Difference (GD) methods.

Results obtained using Fujii's method are shown for (a) N = 25, b N = 50, results obtained using GD's method are shown for (c) N = 25 (d) N = (50); and those obtained using SAM algorithm are shown for (e) N = 25 (f) N = 50.

Fig. 2: Ratio of temporal variance and temporal mean comparisons across frames for different regions of activity.

a Synthetically generated image containing discrete regions with pre-defined speckle activity. b σ/μ vs. number of frames N for region R1. c σ/μ vs. number of frames N for region R2. d σ/μ vs. number of frames N for region R3. e σ/μ vs. number of frames N for region R4. f σ/μ vs. number of frames N for region R5. The blue, orange and green solid lines correspond to σ/μ vs. number of frames N calculated using the Fujii, GD, and SAM algorithms, respectively.

Figure 2b–f depicts the quantitative comparison of each region of activity in the synthetic dataset, by calculating the ratio of temporal variance and the temporal mean (σ/μ) within each region as a function of the number of frames N. It is observed that the SAM algorithm demonstrates the lowest variation as a function of N, which in turn implies higher confidence in the generated activity maps.

Figure 3 depicts the biospeckle activity maps obtained using Fujii, GD, and SAM for the maize and soybean seed dataset. In the case of maize seeds, only the Fujii and SAM maps were able to distinguish between the epithelium and radicle. However, SAM was able to highlight regions of higher biological activity with better contrast as compared with Fujii. For the soybean dataset, SAM yields the highest contrast in comparison to GD and Fujii. SAM is also better at suppressing the noise outside the region of interest (the seed in this case), followed by GD and then Fujii. From both datasets, it is clear SAM preserves more details than Fujii and GD with better contrast. Although the SAM algorithm has an additional overhead of an RGB image for creating the mask, our studies have shown that once the T₀ value is correctly estimated, SAM consistently outperforms other commonly used methods in terms of better noise reduction and contrast estimation.

Fig. 3: Visual temporal activity maps of maize and soybean seeds using different speckle processing algorithms.

a Fujii, b GD, and c SAM algorithms; and activity maps for soybean seeds obtained by using d Fujii, e GD, and f SAM algorithms.

Figures 4a and 5a depict speckle images of viable and non-viable pigeon pea seeds captured after soaking in water for 2 and 8 h, respectively. The top row of all the images in Figs. 4a–d and 5a–d are viable seeds, while the bottom row is images of the non-viable seeds with low germination potential. Figure 5c and d shows the respective GD and SAM outputs estimated from the speckle image sequence and RGB image (in the case of SAM), for peanut and pigeon pea seeds at T = 2 h and T = 8 h, respectively. Fujii-based analysis technique performed poorly with this dataset, hence we have not included its output, and however, the calculated biospeckle activity level has been documented in Tables 1 and 2.

Fig. 4: Images of peanut seeds during the experiment.

a Speckle image captured using a developed prototype of viable peanut seeds and non-viable peanut seeds (bottom row). All seeds have undergone imbibition for timeT = 2 h. b RGB image of the same peanut seeds after being kept for germination for time T = 48 h. c GD output of viable and non-viable peanut seed samples at T = 2 h. d SAM output of peanut seed samples atT = 2 h.

Fig. 5: Speckle images of viable (top row of each image) and non-viable (bottom row of each image) pigeon pea seeds captured using our prototype and their computed temporal activity maps.

a RGB image of seeds after having undergone imbibition for time T = 8hrs (b) RGB image of the same pigeon seed samples after being kept for germination for time T = 48hrs (c) GD output of viable and non-viable pigeon pea seed samples (imbibition time T = 8hrs) (d) SAM output of pigeon pea seed samples (imbibition time T =  8hrs).

Table 1 Normalized biospeckle activity (BA) for peanut seeds

Table 2 Normalized biospeckle activity (BA) for pigeon pea seeds

Furthermore, to verify the applicability of the proposed technique for evaluating seed viability, we also performed seed germination tests. Figures 4b and 5b depict the same seeds after germination at T = 48 h. As was expected seeds in the top row, which were known to be viable, demonstrated radicle emergence, while non-viable seeds in the bottom row with low germination potential, failed to germinate under identical ambient temperature, moisture, and humidity conditions. We compared outcomes using our method with that of the germination tests to ascertain whether the SAM approach could successfully distinguish between viable and non-viable seeds. As seen in Figs. 4d and 5d, there are clear differences in pixel intensities (denoting the level of activity) between viable and non-viable seeds while the same conclusions cannot be drawn from the GD outputs (of peanut and pigeon pea seeds in Fig. 5c and d). Quantitative comparisons of normalized biospeckle activity (maximum value of 100) of seed regions are populated in Tables 1 and 2. The experiments conducted for each category showed mean biospeckle activities between non-viable and viable seeds relatively higher for the SAM technique when compared with Fujii’s and GD. Differentiating viable from non-viable seeds is easier in the case of SAM because the mean biospeckle activity peaks are well separated in comparison. We also performed tests to establish the quality, reliability and statistical significance of our speckle images. The calculated values of Cohen’s d, signal-to-noise ratio (SNR) and t-test values (t-statistic and p-value) were marginally higher for the GD method (1.64, 1.16, 3.67, and 0.0018, respectively) as compared to SAM (1.09, 0.77, 2.45, and 0.025, respectively) indicating that SAM provides higher mean signals but at the cost of increased variability, as reflected in its lower SNR and moderate effect size. GD strikes a balance between strong mean differences and low variability, making it statistically more reliable for distinguishing viable from non-viable seeds. However, Sam exhibits higher normalized difference (ND) values (0.263) compared with GD (0.212) and Fujii (0.118) indicating that SAM provides the largest normalized contrast between viable and non-viable seeds. Thus it can be concluded that the SAM algorithm is a robust technique that can augment the suite of existing biospeckle processing techniques.

Discussion

The selection of superior seeds has a direct impact on crop yields and therefore a positive influence on global food security. Higher crop yields translate into increased availability and affordability of food, improving access to nutrition for communities worldwide. This not only ensures a more stable and balanced diet for individuals but also helps alleviate hunger and malnutrition in regions prone to food scarcity. A careful selection of better seeds constitutes a transformative approach to bolstering crop yields and enhancing global food security. In this work, we have reported on the design and performance of ‘AgriSPEC’, which is a novel, compact, portable smartphone-based embodiment of a biospeckle imaging device that uses minimal components and is easy to assemble and use in field conditions. We have also introduced a new graphical processing technique (‘Speckle Activity Map’ or ‘SAM’) to process captured laser speckle images. The algorithm developed by us to process the images acquired offers two major advantages over existing methods, one being the generation of activity maps with superior contrast thus providing more insights from the recorded images. The other is the capability to process multiple samples simultaneously, thereby increasing the throughput of this imaging modality for agricultural use cases which will be a key driver for increased adoption of this technology. We have performed a quantitative and qualitative comparison study of our technique against other existing methods and have demonstrated its advantages over them. In this study, the mobile phone (along with the custom attachment) has been used for speckle image acquisition and the images have been subsequently processed on a personal computer. Although our algorithm has not been explicitly run on the smartphone platform for the results reported in this manuscript, the architecture of the algorithm is not complicated and can be easily implemented using the computing capabilities of currently available mobile phones. By leveraging the portability and ubiquity of smartphones, our proposed methodology offers a promising solution to overcome the cost and portability limitations associated with traditional seed grading techniques. Through our research, we aim to contribute to the advancement of seed grading technology, thereby facilitating the widespread adoption of efficient and accurate methods for improving crop productivity and addressing global food security challenges.

Methods

For our experiments, two types of seeds were considered: the GKVK5 variety of Peanut (Arachis hypogaea) and the BRG5 variety of red pigeon pea (Cajanus cajan), as shown in Fig. 6. Samples were randomly selected from viable (<2 months old) and non-viable (more than 12 months old) categories of both varieties. As per standard protocol, the selected seed samples underwent a disinfection process using a 5% sodium hypochlorite solution, followed by sterilization with distilled water. The duration of water imbibition varied depending on the seed type. Peanut seeds were soaked in water for 2 h, whereas pigeon pea seeds were soaked for 8 h. Afterward, the seed samples were gently swabbed using a paper towel and analyzed using our smartphone-based portable setup.

Fig. 6: RGB images of viable and non viable seeds during experiment.

a RGB images of peanut samples (Arachis hypogaea) and b RGB images of pigeon pea samples (Cajanus cajan). Seeds labeled 1 and 2 correspond to viable seeds; seeds labeled 3 and 4 are non-viable seeds.

Figure 6a and b shows the RGB images of peanut and pigeon pea seeds, respectively, selected for our study. Seeds labeled as 1 and 2 in Fig. 6a correspond to viable seeds, while 3 and 4 represent non-viable seeds for peanut samples. The same convention is followed for pigeon pea seed samples in Fig. 6b, with labels 1 and 2 denoting viable seeds and 3 and 4 denoting non-viable seeds. The images were acquired after soaking them into water for 2 and 8 h, respectively.

Figure 7 depicts the AgriSPEC laser biospeckle imager which comprised a custom-designed, 3D-printed housing affixed to the rear face of a smartphone (Samsung A33, 6 GB RAM, 128 GB ROM, 13MP Front camera, and Exynos 1280 Processor). The housing has been designed to hold a red laser diode emitting 5 mW at 650 nm (AVS COMPONENTS) and a white LED (to serve as an auxiliary light source alongside the phone’s in-built flash for capturing RGB images of the sample). We removed the cylindrical lens that came with the laser diode to expand the illumination. The housing also incorporated a microcontroller board (Seeeduino Xiao SAMD21) for enabling communication with the smartphone using the USB CDC protocol via its USB port. The white LED was powered through Seeeduino’s GPIO pins. However, as the power from the microcontroller was insufficient for driving the laser diode, additional transistor circuitry was introduced for its operation (Fig. 8).

Fig. 7

Smartphone-based biospeckle imager.

Fig. 8

Block diagram of portable, smartphone-based biospeckle imaging device.

To illuminate the sample, we developed a custom Android application to control the operation of the LED and the laser diode. The application was programmed to switch the LED to its ‘ON’ state, capture an RGB image of the sample, and switch to its OFF state after which, the laser diode was switched on and a speckle video sequence was captured over a ‘40’-s duration. The focus was adjusted manually, the shutter speed was set to 1/125th second, the ISO was fixed at 50, and the zoom magnification was kept at ×3. The 40 s-long videos at 30 fps were recorded to ensure that we could obtain image sequences that contained at least 1000 frames.

In order to validate the performance of our device, we performed germination tests for our samples which is the standard procedure for testing the viability of seeds. The purpose of germination testing is to analyze the potential of seeds to grow into a healthy plant under the appropriate temperature, moisture, and light conditions. To perform this test, both viable and non-viable seeds of the target species (pre-certified by the Indian Council of Agricultural Research) were carefully selected. Seeds from both groups were placed between two layers of wet germination papers in 12 mm Petri plates. The petri plates containing seeds were maintained in a suitable environment under appropriate temperature and moisture conditions to initiate the germination process. During germination, seeds were carefully examined to maintain sufficient water to complete the germination process. Seeds were subjected to a 48-h germination period, during which the emergence of shoots was meticulously observed. Continuous monitoring was conducted to track any developmental changes, and imaging analysis was subsequently performed to validate and document the results. After 48 h (about 2 days), the same seed samples were imaged again to record the emergence of shoots, thereby validating our findings.

There are several well-known visual analysis techniques for the calculation of biospeckle activity from image sequences, such as Fujii’s¹⁹, the generalized difference (GD)¹⁸), the weighted generalized difference (WGD), structure function (SF), and the modified structure function (MSF)²². Fujii’s method uses localized absolute differences weighted by the sum of intensities. As a result, variations above certain pixel intensities tend to be nullified. To address this issue, the GD approach was introduced which estimated the absolute difference of all the combinations of speckle frames. However, the analysis technique can sometimes suffer from an artifact where noise is treated as a signal during computation. The WGD method was introduced to mitigate the effects of signal noise by adopting a ‘moving window’ approach. In this method, the window sizes have to be tried out arbitrarily if the physical properties of the sample under inspection and the imaging setup are unknown. The Fujii and GD methods remain the most notable of all of these techniques as they are most commonly applied for biospeckle analysis across diverse contexts. However, the performance of these methods is considerably hampered by issues such as insufficient contrasts and sensitivities, which reduces the accuracy and resolution of the resultant activity maps and presents serious challenges to the effectiveness of these techniques. In view of this, we have developed an algorithm titled ‘Speckle Activity Map’ (SAM) which outperforms existing techniques in terms of the overall contrast of the activity maps generated for analyzing the biospeckle activity. Below, we describe the steps involved in implementing SAM:

The first step in the SAM algorithm requires the creation of a mask to separate the sample region (region of interest (ROI)) from the background. To create the mask, an RGB image of the sample was captured with a dark background. The Otsu’s thresholding algorithm was applied on the red channel to discriminate between the ROI and non-ROI sections of the image. The regions containing the sample were assigned a value equal to 1 while the background regions were assigned a value equaling 0. Figure 9a shows the input RGB image of the peanut seeds and its respective image mask is presented in Fig. 9b.

Fig. 9: RGB image of peanut seeds and their respective masks.

a RGB image of peanut seed samples and b corresponding image masks generated using Otsu’s method.

Our first set of validations was performed on publicly available biospeckle datasets²³ which did not contain any RGB images of the sample hence, the masks had to be created manually by drawing contours around the Fujii or GD output which can be done using standard image processing software packages (Gimp, Adobe Photoshop, etc). Figure 10a depicts a processed activity map of a maize seed and Fig. 10b shows the generated mask. Biospeckle images often encounter unwanted artifacts caused by inherent instabilities in experimental setups, including vibrations and fluctuations in laser source intensity. These variations negatively impact the quality of activity maps and thus require suppression of these artifacts through appropriate image processing techniques to enhance the accuracy of the analysis. To address this, we carefully optimized a threshold parameter, denoted as T₀, to mitigate these artifacts. The following steps outline the procedure, as depicted in Fig. 11.

• 1. We consider only the red-channel of each frame in the recorded image stack (${I}_{0}$ to ${I}_{N}$, where, N is the total number of images in an image stack) and calculate the absolute difference between the resultant adjacent frames.

• 2. Assign an arbitrary value to ${T}_{0}$ ranging between 1 and 30. This value is an initial value that will need to be further optimized by following the procedures below.

• 3. If the value of a pixel in the images after absolute difference (D₀ and D_N−1)) exceeds T₀, set it to 1; otherwise, keep it as 0.

• 4. Add the values of each pixel as it varies over time (across all frames) to obtain the matrix S, which is then normalized to derive ${S}_{{{\rm {norm}}}}$ (refer to Fig. 11). S_norm is obtained by dividing each element of the matrix $S$ by the maximum value of $S$ (given by ${S}_{\max }$).

• 5. Multiply ${S}_{{{\rm {norm}}}}$ with the mask created using the RGB image to obtain ${S^{\prime} }_{{{\rm {norm}}}}$.

• 6. A typical histogram plot of ${S^{\prime} }_{{{\rm {norm}}}}$ will span 0 to 1, exhibiting a prominent peak at zero and a secondary peak in close proximity to 0.5. Figure 12a and b show optimal histogram plots with soybean and peanut seed data, respectively. To arrive at the ${T}_{0}$ value, which will eventually yield the best histogram (described in 6), we maximize the convolution of (the histogram of) ${S^{\prime} }_{{{\rm {norm}}}}$ and H by iteratively assigning values between 0 and 30. The ${T}_{0}$ value which satisfies the above will yield the optimal image histogram H.

Fig. 10: Activity maps of soybean seed and the generated RGB mask.

a Activity map generated using the generalized difference (GD) method for maize seed dataset and b its corresponding image mask.

Fig. 11

Workflow depicting the steps involved in choosing the optimal threshold parameter T₀.

An image histogram (H) is a graphical representation of its intensity distribution, i.e., it depicts the number of pixels for each intensity value. For an image with good contrast, the most frequently occurring intensity values are spread out or ‘stretched’ out over the total intensity range of the image (in computer vision, this is commonly known as ‘histogram equalization’). For our speckle images, the histograms (whose x-values range between 0 and 1) contain a primary peak and a smaller, secondary peak (as shown in Fig. 11a and b). For images corresponding to optimal contrast, the secondary peak is typically centered at ~0.5. If the secondary peak occurs closer to 1, then regions of high activity get amplified whereas low-activity regions get suppressed. If the secondary peak occurs closer to 0, the reverse happens, i.e., regions of low activity get enhanced whereas regions of higher activity become attenuated. Therefore, histograms that satisfy this optimal condition are considered as ‘ideal’ histograms for our purpose. To arrive at an optimal threshold ${T}_{0}$ (step #6), its value is iterated from 0 to 30 (this range was set based on empirical observations) till the convolution of (the histogram of) ${S^{\prime} }_{{norm}}$ and H (given by C) is maximized. The ${T}_{0}$ value which maximizes the convolution is taken to the threshold value for subsequently processing the images.

Fig. 12: Optimal histogram plots obtained for SAM outputs.

a Histograms for soybean seed samples. b Histograms for peanut seed samples.

In the next section, we benchmark the SAM algorithm with existing techniques based on contrast metrics. We also numerically quantify the biospeckle activity (BA) which involves calculating the average pixel intensity in both the x and y directions. However, the average pixel intensities are specifically calculated within the seed region by masking the background and selecting the seed area only. The mathematical representation for computing normalized biospeckle activity (BA) is expressed as

$${{\rm{Biospeckle}}\; {\rm{Activity}}}\left({\rm{BA}}\right)=\sum {{\rm {SAM}}}(x,y)/{N}_{x}{\times N}_{y}$$

(1)

where N_x and N_y are the number of pixels in each direction. Similarly, the biospeckle activity within the active seed region is determined using standard procedures, including GD and Fujii methods, for comparative analysis of the results.