Fig. 1: Principles of the proposed spectral convolutional neural network (SCNN).
From: Spectral convolutional neural network chip for in-sensor edge computing of incoherent natural light
a Existing optical neural networks (ONNs) are based on coherent light sources for computing. They are incapable of broadband light field sensing and in-sensor computing. b In our design, we implemented an SCNN by integrating very large-scale spectral filters on CMOS image sensor (CIS). Our SCNN can accept incoherent natural light and perform analog 2D convolution calculations directly. c The metasurface-based optical convolutional layer (OCL) integrates pixel-aligned metasurface units on a CIS. d The pigment-based OCL is fabricated by lithography on a 12-inch wafer. e The working principles of our OCL. One OCL contains an \(H\times W\) array of identical OCUs and each OCU has \(K\) convolutional kernels, resulting in calculation results of size \(H\times W\times K\). \({{{{\bf{x}}}}}_{N}\): The input spectral signal. \({{{{\bf{w}}}}}_{{KN}}\): Transmission response of the spectral filter. \({I}_{KN}\): Photocurrent of the CIS pixel.
