Fig. 1: An optical vector-vector dot product multiplier for characterizing the optical energy consumption of an ONN.

a The role of optical vector-vector dot products in executing the forward-pass operation in a fully connected neural network. The weighted sums of neural activations are performed optically (shaded area) and the element-wise nonlinear activation functions are performed electronically. Each neuron in the middle (hidden) layer is color-coded to show the correspondence to their representations in (b). The shaded area in the neural-network schematic illustrates the neurons and weights involved in one dot product. b A step-by-step illustration of the computation of optical vector-vector-dot product between \(\mathop{{{{{{\bf{x}}}}}}}\limits^{ \rightharpoonup }\) and \({\mathop{{{{{{\bf{w}}}}}}}\limits^{ \rightharpoonup }}_{{{{{{\bf{1}}}}}}}\). The top row shows mathematically abstract operations, and the bottom row shows the corresponding physical operations with optics. “\(\circ\)” denotes element-wise multiplication between two vectors of the same size. c An illustration of how the optical fan-in operation allows less-than-1-photon-per-scalar multiplication when the vector size is large. A single lens is used to sum the intensities of the spatial modes encoding the element-wise products onto a detector. For sufficiently large vector size N, even if each individual spatial mode contains \(\epsilon \, < \, 1\) photon on average, the total number of photons impinging on the detector \(\epsilon N\) will be ››1.