Introduction

Silicon nitride (SiN) has enabled a wide range of applications, including sensing1,2,3, optical communications4, interconnects5,6, spectroscopy7,8, light detection and ranging9,10, and quantum information processing11,12, due to its broad transparency window and accessibility. Its impact also results from the ultra-low waveguide propagation losses it can provide13, with losses below 0.1 dB m−1, facilitating the management of high optical power and enabling nonlinear applications such as optical frequency comb generation14,15. The small thermo-optic coefficient of SiN and silicon dioxide (SiO2) results in a low wavelength dependence on temperature16. This property benefits passive devices but presents challenges for tunable components. Additionally, the relatively low refractive index contrast between SiN and SiO2 minimizes phase errors in devices such as interferometers, enhancing robustness against fabrication imperfections17. However, this advantage comes at the expense of larger device footprints, which limits the number of components that can be integrated per chip. For instance, in a platform where the SiN is 400 nm thick, a 850 nm wide waveguide should be routed with a minimum bend radius of 50 μm to achieve negligible losses18.

Silicon photonics is transitioning from the era of medium-scale integration (MSI; 10-500 components) to that of large-scale integration (LSI; 500–10000 components), driven in part by the demands of photonic quantum computing19. The rise of co-packaged optics (CPO) demands LSI photonics—and eventually very-large-scale integration (VLSI) chips with more than 10,000 components—offering high bandwidth and connectivity20, alongside high-performance electronic chips within the same package. This level of integration is essential for enabling the next generation of high-performance computing systems, including those driving artificial intelligence21. For SiN to play a key role in co-packaged optics (CPO) and VLSI photonic circuits, substantial reductions in component sizes, waveguide routing bends, and input/output coupler dimensions are required.

Spatial and mode multiplexers, along with polarization management devices, are essential components in many applications22,23,24, especially in optical communications25. Recently, various low-loss multiplexers and polarization splitters implemented on a silicon nitride platform have been reported26,27,28,29,30,31. However, most demonstrations require a substantial footprint, and attempts to miniaturize these devices often introduce additional fabrication complexity, such as deep etch steps or the integration of additional material layers32,33. SiN multiplexers can be implemented using gratings34,35, cascaded Mach-Zehnder interferometers (MZI)36, ring resonators37, and directional couplers26. Gao et al. demonstrated a coarse wavelength division multiplexer (CWDM) for the O-band with a footprint of 1 × 0.6 mm2, achieving a minimum loss of 1.55 dB38. More recently, a three-mode mode division multiplexer (MDM) with a length of over 1.4 mm operating at visible wavelengths was demonstrated, achieving an insertion loss of 3.5 dB for the highest-order mode23. Kudalippalliyalil et al. presented a compact polarization beam splitter (PBS) measuring 80 × 13 μm2 on a 400 nm thick SiN platform, achieving an efficiency of up to  −0.7 dB and a crosstalk level below  −18.0 dB24.

In this work, we demonstrate the potential of the inverse design technique to improve component integration density on photonic platforms with moderate refractive index contrast by presenting three ultra-compact freeform devices—a four-channel CWDM, a five-mode MDM, and a PBS—fabricated on a single-etch SiN platform. The four-channel CWDM, with a footprint of 24 × 24 μm2, achieves a maximum measured efficiency of  −0.8 dB at the central wavelength across all channels. The five-mode MDM, with a compact 16 μm length, achieves an average maximum efficiency of  −1.0 dB for the TE00 mode and  of −2.3 dB for the TE04 mode, with crosstalk below  −11.0 dB. The polarization beam splitter, measuring 24 × 12 μm2, exhibits an average efficiency of  −0.8 dB for the TE mode and of  −0.9 dB for the TM mode across the entire C-band, with crosstalk as low as  −18.0 dB. These devices feature the smallest reported footprints among similar demonstrated devices, significantly enhancing integration density while maintaining high performance.

Results

All the proposed devices were implemented using a fully etched SiN core (400 nm thick) surrounded by SiO2. The refractive indices used for the design were 1.997 for SiN and 1.441 for SiO2 at 1550 nm. For the inverse design process, we employed the Python-based wrapper LumOpt, which enables density-based topology optimization while incorporating fabrication constraints such as a minimum feature size and a smoothing filter with a radius of up to 160 nm. To further improve the performance of the four-channel CWDM, a deep machine learning model from PreFab Photonics39, optimized for the SiN fabrication process offered by Applied Nanotools (ANT), was used to refine the layout and anticipate fabrication deviations. The devices were fabricated using electron-beam lithography (EBL) and characterized with a wafer-level automated test station (see “Methods”).

Coarse wavelength division multiplexer

The compact CWDM was designed using the fundamental TE mode of the input waveguide as the input mode for the inverse design process, while the fundamental TE modes of the four output waveguides were used as the output modes. The waveguide width was set to 850 nm to ensure single-mode operation. The target central wavelengths were 1530 nm, 1550 nm, 1570 nm, and 1590 nm, and the optimization was carried out using five wavelengths within a 10 nm bandwidth around each central wavelength. Figure 1 presents the original device design, a scanning electron microscopy (SEM) image of the fabricated original device with a footprint of 24 × 24 μm2, and the simulated electromagnetic energy density at the central wavelength of each channel. At 1530 nm, the light undergoes a directional change before propagating through the upper channel. In contrast, for the other channels, the light follows a relatively smooth path towards their respective output (see Supplementary Movie).

Fig. 1: Four-channel coarse wavelength division multiplexer.
figure 1

a Device design, where the black areas represent silicon nitride and the white ones represent silicon dioxide. b SEM image of the fabricated device without layout corrections, with a total footprint of 24 × 24 μm2. c Simulated electromagnetic energy density at the central operating wavelength of each channel.

Full size image

The simulated and measured transmission results for the original and refined designs are shown in Fig. 2. Simulated transmission and crosstalk were  −0.9 dB and  −18.4 dB at 1530 nm,  −0.7 dB and  − 17.4 dB at 1550 nm,  −0.5 dB and  −14.7 dB at 1570 nm, and  −0.7 dB and  −20.1 dB at 1590 nm. Simulations also indicated that back-reflection into the input waveguide remained consistently below  −16.9 dB at the central wavelengths. Measurements from three identically fabricated devices without layout refinement, shown in Fig. 2b, demonstrated highly repeatable performance. The average measured transmission was  −1.7 dB at 1511.3 nm,  −1.3 dB at 1531.3 nm,  −1.0 dB at 1551.3 nm, and  −1.0 dB at 1571.3 nm, with crosstalk remaining below  −11.6 dB at these wavelengths. The discrepancies between simulated and measured results are mainly attributed to over-etching during fabrication (see Supplementary Fig. 4).

Fig. 2: Simulated and measured transmission efficiency of the four-channel coarse wavelength division multiplexer.
figure 2

a Simulated transmission obtained using a 3D finite-difference time-domain algorithm. Measured transmission from fabricated devices: (b) original design and (c) layout corrected design. Solid lines represent average transmission, and gray shaded areas show the range between minimum and maximum values obtained during measurements.

Full size image

Figure 2 c shows the results for two identically fabricated devices with layout refinement. The deep machine learning model adds silicon nitride where material loss is predicted and removes it where excess SiN is expected after fabrication, thereby bringing the fabricated device response closer to the simulated design (see Supplementary Note 3). The average measured transmission was  −1.2 dB at 1517 nm,  −1.2 dB at 1537 nm,  −1.5 dB at 1557 nm, and  −0.8 dB at 1577 nm, with measured crosstalk values of  −19.4 dB,  −20.2 dB,  −14.7 dB, and  −25.2 dB, respectively. The refined design showed slight improvements in transmission efficiency for channels 1, 2, and 4, and a small drop for channel 3. The most notable enhancements were the correction of the wavelength shifts and a reduction in crosstalk. The wavelength shifts decreased by more than 5 nm for all central wavelengths, whereas crosstalk improved by more than 5 dB for channel 1, with improvements across all channels.

Mode division multiplexer

A compact five-mode MDM multiplexer was optimized using the fundamental TE mode of five 850 nm-wide input waveguides, spaced 1.2 μm apart, as the input modes, while the first five TE modes of a 4 μm-wide output waveguide served as the output modes in the inverse design process. Broadband optimization, spanning the entire C- and L-bands, was performed over a rectangular design region of 16 × 7 μm2, with a minimum feature size and a smoothing filter radius of 120 nm applied during the optimization process. Figure 3 presents the final device design, including the routing network, a SEM image of the fabricated structure (with an inset showing the region near the input waveguides), and the simulated electromagnetic energy density at 1550 nm for each mode. The five-mode MDM multiplexer routes the fundamental TE00 mode of the input single-mode waveguides to the TE00 (channel 2), TE01 (channel 3), TE02 (channel 4), TE03 (channel 1), and TE04 (channel 5) modes of the output multimode waveguide. Additionally, 16 μm radius freeform bends were used to route channels 1 and 518, minimizing crosstalk. Furthermore, the five-mode MDM can be used to implement an micro-electromechanical systems (MEMS) inter-chip modal switch40 if instead of using five fixed input waveguides only one input, mounted on a MEMS actuator, is used to select which mode is excited.

Fig. 3: Five-mode mode division multiplexer.
figure 3

a Device design and routing network, where the black areas represent silicon nitride and the white ones represent silicon dioxide. b SEM image of the fabricated device with a total footprint of 16 × 7 μm2, including an inset showing the input waveguides. c Simulated electromagnetic energy density for each channel at 1550 nm.

Full size image

The simulated and measured transmission results are shown in Fig. 4. The average simulated transmission efficiency across the entire C-band is  −0.9 dB for TE00,  −0.9 dB for TE01,  −1.2 dB for TE02,  −1.3 dB for TE03, and  −1.3 dB for the TE04 mode, respectively. The 3-dB bandwidth exceeds 120 nm, and the average crosstalk across these wavelengths remains below  −12.3 dB for all channels. The measured average efficiency in the C-band is  −1.0 dB for TE00,  −1.1 dB for TE01,  −1.3 dB for TE02,  −1.9 dB for TE03, and  −2.3 dB for TE04, with crosstalk values of  −11.0 dB,  −10.6 dB,  −13.7 dB,  −11.1 dB, and  − 16.1 dB, respectively. Furthermore, the transmission spectra is flat across the entire C-band. The five-mode MDM multiplexer, including its routing network, maintains efficiencies comparable to those of its silicon counterparts while preserving its ultra-compact footprint41. Modal crosstalk can be reduced by increasing the channel spacing in the five-mode MDM multiplexer; however, this would also increase the switching time when used as part of a MEMS switch.

Fig. 4: Simulated and measured transmission efficiency of the five-mode mode division multiplexer.
figure 4

a Simulated transmission obtained using a 3D finite-difference time-domain algorithm. b Measured transmission of the fabricated device for all five channels.

Full size image

Polarization beam splitter

Figure 5 presents the ultra-compact PBS design, an SEM image of the fabricated device, and the simulated energy density at 1550 nm for both TE and TM polarizations. The PBS was designed using the fundamental TE and TM modes of the input waveguide as input modes for the inverse design process, with the TE and TM modes directed to output channels 1 and 2, respectively. The spacing between the output waveguides was set to 4.925 μm, and both the input and output waveguide widths were set to 850 nm. The optimization was performed within a 24 × 12 μm2 region, targeting broadband operation spanning from 1520 nm to 1630 nm, and incorporating a minimum feature size and a smoothing filter with a radius of 160 nm.

Fig. 5: Compact polarization beam splitter.
figure 5

a Device design, where black areas represent silicon nitride and white areas represent silicon dioxide. b SEM image of the fabricated device, which has a total footprint of 24 × 12 μm2. c Simulated electromagnetic energy density at 1550 nm for both the TE and TM polarizations.

Full size image

The simulated and measured transmission spectra of the compact PBS device for channels 1 and 2, corresponding to TE and TM polarizations, respectively, are shown in Fig. 6. The average simulated TE transmission efficiency to channel 1 is  −0.3 dB, while the undesired TM transmission to the same channel averages  −20.1 dB across the C-band. Similarly, the average simulated TM transmission efficiency to channel 2 is  − 0.3 dB, with the undesired TE transmission averaging  −25.8 dB. These values indicate a simulated polarization extinction ratio (PER) exceeding 19.8 dB. The measured average TE transmission efficiency to channel 1 is  −0.8 dB, while the average TM transmission efficiency to channel 2 is  −0.9 dB. The undesired TM transmission to channel 1 and the undesired TE transmission to channel 2 are  −15.5 dB and  −15.3 dB, respectively. Measurements were limited to the C-band due to constraints imposed by the TM grating coupler (see “Methods”). The reduction of the experimental PER to ~14.5 dB is attributed to fabrication variations, including over and under etching (see Supplementary Fig. 6).

Fig. 6: Simulated and measured transmission efficiency of the polarization beam splitter.
figure 6

a Simulated transmission from 1525 nm to 1625 nm. b Measured transmission in the C-band for the fabricated device for both the TE and TM polarizations. In both cases, the right subplot represents the transmission of channel 2, while the left subplot represents the transmission of channel 1.

Full size image

Although it may be desirable, most inverse-designed photonic structures do not exhibit periodic, regular, or modular patterns and therefore cannot be easily interpreted or modeled using traditional physical models. However, the devices presented in this work—including the four-channel CWDM, the five-mode MDM, and the PBS—can be understood as systems that precisely engineer light-matter interactions at the subwavelength scale. Through their non-intuitive, freeform geometries, these devices spatially tailor the effective refractive index to produce highly specific patterns of constructive and destructive interferences, each optimized for particular wavelengths, spatial modes, or polarizations. Thus, the dispersion required to separate wavelengths, polarizations or to manipulate a wavefront to convert it into a different spatial mode is achieved by engineering the refractive index of the subwavelength pixels. Therefore, this approach provides capabilities that go far beyond conventional photonic components.

Discussion

The inverse design technique has been increasingly utilized in both academia and industry to enhance component density on silicon-on-insulator (SOI) platforms. Many passive and active devices have been successfully miniaturized using this technique on platforms with high refractive index contrast42. However, inverse design has been less explored for platforms with medium to low index contrast, such as silicon nitride, with only a few exceptions, including reflectors43, TM polarizers44, mode converters45, and beam splitters46. This is primarily due to the significantly higher computational effort required when working with large optimization regions.

It is important for inverse-designed devices to demonstrate good fabrication repeatability and to remain robust against fabrication variations while maintaining high performance. Our freeform four-channel 24 × 24 μm2 SiN CWDM achieves high transmission efficiency, ranging from  −0.8 dB to  −1.5 dB at the central wavelength of each channel. These results are comparable to those of a freeform four-channel 14 × 16 μm2 silicon CWDM device, which demonstrates efficiency levels between  −2.0 dB and  −3.3 dB47. Additionally, the unique five-mode freeform SiN MDM device has a remarkably small footprint of 16 × 7 μm2, compared to SiN phase-matching multiplexers, which typically occupy areas greater than 1 mm2, achieving transmission efficiencies from  − 1.0 dB to  − 2.3 dB. This footprint is comparable to the only freeform five-mode MDM in silicon presented so far in the literature, which has a footprint of 6 × 10 μm2 and achieves a transmission efficiency ranging from  −2.4 dB to  −3.9 dB41. Our inverse-designed SiN multiplexers have a compact footprint and achieve performance on par with, or better than, their silicon counterparts, despite the lower index contrast. While our CWDM design demonstrates high efficiency and good fabrication repeatability, the central wavelengths shifted toward lower values even after applying a layout refinement, primarily due to over-etching during fabrication. Fabrication variations also affect crosstalk more than transmission efficiency in broadband devices—including the MDM and PBS—with the effect particularly pronounced in the PBS. These issues can be mitigated using machine learning-based correction models48, which continue to improve, as demonstrated with the CWDM device, ensuring that miniaturization does not compromise performance.

In summary, we demonstrated experimentally three ultra-compact freeform silicon nitride devices—a four-channel CWDM, a five-mode MDM, and a PBS—using the inverse design technique. These devices achieve performance levels that have not been demonstrated before in such small SiN structures. For instance, the CWDM device demonstrates a size reduction of more than 1200× compared to traditional cascaded MZI silicon nitride designs38. Our results show that applying inverse design techniques to the silicon nitride platform can enable the development of highly compact devices while preserving superior passive performance, paving the way for large-scale integration.

Methods

Optimization

To design the devices, we use the Python-based wrapper LumOpt, which supports continuous density-based topology optimization and employs the Ansys Lumerical 3D finite-difference time-domain (FDTD) algorithm to solve the direct and adjoint problems for sensitivity analysis. The optimization is driven by gradient-based algorithms from SciPy.

The topology optimization workflow begins with an initialization step, in which the optimization volume and the initial core and cladding refractive indices are defined. LumOpt uses a rectilinear optimization grid that is fully aligned with the 3D FDTD simulation grid used for both the direct and adjoint solves. For each device, we evaluated three initial conditions: the design region was uniformly initialized with either silicon nitride, silicon dioxide, or a refractive index equal to the average of the two. For the CWDM and PBS devices, the best results were obtained with the average-index initialization, whereas for the MDM device, initializing with silicon nitride yielded superior performance.

The optimization consists of three stages. In the first phase (Greyscale optimization), the material distribution varies continuously between the core and cladding indices. In the second phase (Binarization), the design is forced to take on discrete values corresponding to either the core or cladding material. A convolution filter and a projection function are employed to smooth the design region. Specifically, a circular top-hat kernel with a radius of 120 or 160 nm is used to eliminate small holes, sharp corners, and other non-manufacturable features. Simultaneously, an approximation of the Heaviside function transforms the linear mapping of the design parameters into a near step-like function, controlled by the steepness parameter β. As β → ∞, the geometry becomes increasingly binary. This step may introduce sharp peaks in the figure of merit, which are mitigated through additional optimization iterations to stabilize the design. The final phase (Design for the manufacturing (DFM)) introduces explicit minimum length-scale geometric constraints to eliminate residual small features and ensure fabrication feasibility. An overview of the topology optimization process is provided in Supplementary Note 4.

The optimization process was performed on a single computer equipped with an Intel Core i9-12900 processor and 64 GB of RAM. The computational time per iteration varied for each device: for the CWDM, it ranged from 50 min to ~5 h; for the MDM, from 20 min to about 5.5 h; and for the PBS, from 15 min to ~2.5 h. The final optimization phase was typically the most time-consuming. Supplementary Figs. 1, 2, and 3 illustrate the complete optimization trajectory for each device, along with the material index distribution at representative iterations of each optimization stage.

Fabrication

The photonic integration platform used in this work is based on silicon nitride and consists of a 400 nm SiN core layer, a 4.5 μm buried oxide layer, a 3 μm top silicon dioxide cladding, and a 525 μm-thick silicon substrate. All devices were fabricated through multi-project wafer runs at ANT.

An hydrogen silsesquioxane (HSQ) hard mask was spin-coated onto the SiN layer, and the device patterns were written using 100 keV EBL. The exposed HSQ resist was developed using a tetramethylammonium hydroxide-based developer. Pattern transfer into the SiN layer was carried out via a single-step anisotropic reactive ion etching process using a CHF3/O2 chemistry. After etching, a 3 μm top cladding of silicon dioxide was deposited using plasma-enhanced chemical vapor deposition49. Although the EBL process enables feature sizes as small as 120 nm, a minimum feature size and spacing of 160 nm was enforced for the CWDM and PBS devices to improve fabrication robustness and ensure compatibility with large-scale complementary metal-oxide-semiconductor (CMOS) foundries using 193 nm deep ultraviolet lithography.

The fabricated waveguides exhibit an average sidewall angle of ~83. 5°, with measured propagation losses of about 1 dB cm−1 for the TE mode at 1550 nm under single-mode operation.

Measurement

The devices were characterized using an EXFO OPAL-EC wafer-level automated test station, which features a motorized high-resolution 4-axis motion system with thermal control, where the chips were placed during measurements. The station also includes a 30° polarization-maintaining 12-fiber array mounted on a hexapod, connected via polarization-maintaining fibers to an EXFO T100S-HP tunable laser and a CTP10 passive optical component testing platform for transmission efficiency measurements. For TM measurements, a polarization rotating fiber was inserted between the laser and the fiber array. A Z-profile scan was conducted on the chip surface to maintain a consistent coupling distance of ~20 μm throughout all TE measurements. To ensure accurate and repeatable device efficiency measurements, surface coupling optimization was performed using the same automated alignment sequence for all devices. Transmission measurements were then normalized against reference surface grating couplers connected in a loopback adjacent to each device, eliminating variations due to coupling and waveguide propagation losses and enabling a direct evaluation of device performance. In addition, to characterize the inverse-designed MDM, we used an auxiliary adiabatic directional coupler MDM placed back-to-back with the device under test. The two MDMs were connected via a 38 μm-long, 4 μm-wide multimode waveguide. To normalize the measurements and isolate the performance of the inverse-designed MDM, we fabricated a nearby reference structure consisting of two identical directional coupler MDMs connected by a multimode waveguide with the same length. By comparing the transmission spectra of the inverse-designed MDM structure and the reference structure, we extracted the loss and crosstalk attributable to the inverse-designed MDM alone. It is worth noting that the directional coupler MDM has a length of ~800 μm, highlighting the substantial footprint reduction achieved by the inverse-designed MDM.

The grating coupler used for the TE measurements was a non-uniform design optimized using SPINS-B50. With a minimum feature size of 400 nm, it achieved an experimental minimum coupling loss of 12.1 dB through a single-pitch loopback configuration. The coupling angle for TE measurements was maintained at 30°, aligning with the fiber array. For TM measurements, a regular grating with a period of 1.88 μm and a fill factor of 0.5 was used, resulting in an experimental minimum coupling loss of 19.7 dB. A 26° coupling angle was employed, increasing the coupling distance to over 100 μm. This distance was maintained consistently across all TM measurements to ensure repeatability. The bandwidth of the TM grating coupler was limited to the C-band, as TE crosstalk became significant in the L-band.