Extended Data Fig. 5: SpatialData facilitates the preparation of datasets for deep learning applications and it integrates with existing deep learning ecosystems.

(a) Building on the query interface, SpatialData allows to generate PyTorch datasets that represent tiles of the original SpatialData. Shown is an example use case, using tiles centered on cells to train a DenseNet encoder model for supervised cell-type prediction. The specific model architecture, without weights, is provided by the MONAI framework, and this example shows how we can readily interface with existing deep learning ecosystems. (b) The effective definition of deep learning datasets can harness common coordinate systems to allow for the combination of different spatially aligned elements. Shown are H&E image and Xenium replicate 1 aligned datasets precedently introduced in main text Fig. 2a. (c) Enlarged view of a subset of the two datasets, overlaying the cells from Xenium, colored by cell type, to the H&E image from Xenium. SpatialData allows to extract image tiles of the desired resolution (here 32x32 pixels) around the Xenium cells. (d) The tiling extraction process takes advantage of the multiscale representation and the chunked Zarr storage for efficient memory usage. The first allows the extraction of the tiles from the appropriate (downscaled) resolution, the second ensures that only the data chunk(s) containing the information about the tiles are loaded from disk. Note: the 500x and 1000x downscaling factors and the size of the chunks have been chosen for illustrative purposes. (e) Visualization of cell-type labels predicted by the model. Note: due to the illustrative purpose of this example, focusing on the demonstration of the infrastructure, network training has been limited to a small number of epochs, and systematic hyperparameter optimization has been omitted. This is reflected in the suboptimal accuracy of the predictions. The full example can be found in the online documentation (https://spatialdata.scverse.org/en/latest/tutorials/notebooks/notebooks/examples/densenet.html).