Fig. 4: Reconfigurable in-sensor image preprocessing and spectral classification.

a Projected original image and corresponding image acquired directly by the sensor. The intensity of image is related to PCR. Image preprocessing with three different operations, b Gaussian filtering, c Edge enhancement, d Image sharpening. The operating kernels modulate the PCR of each pixel by independently changing the bias current. Blue/red squares represent positive/negative bias currents and white square represents no bias current. Colors from light to saturated represent increasing absolute values of bias currents. Experimental results of distinct operations are compared with simulations. e, f PCR as a function of bias current. e wavelengths from 405 to 1550 nm. f wavelengths from 1490 to 1590 nm. g Convolution kernel for wavelength classification and corresponding PCR. S1-S5 represent the total counts of five pixels per row. After normalizing S1-S5, set the value greater than 0.4 to 1 and the value less than 0.4 to 0, so that S1–S5 = 11111 corresponds to 405 nm, S1–S5 = 01111 corresponds to 650 nm, S1–S5 = 00111 corresponds to 1064 nm, S1–S5 = 00011 corresponds to 1310 nm, S1–S5 = 00001 corresponds to 1550 nm.