Fig. 2: The simulation procedure and the performance of INSTINCT in simulated scenarios.

a The predefined template comprises five spatial domains, named by integers 0 to 4, each possesses a major cell-type within cell-type 0 to 4. Three simulated single-cell datasets with batch effects are generated by simCAS, each used to generate one spatial slice. Initially, each location in all slice is assigned with a cell-type, and a cell belongs to this cell-type in the corresponding single-cell dataset is randomly selected and allocated to the location to form three cell-level slices with single-cell resolution. Subsequently, with \(3\times 3\) region defined as a spot, the three slices are merged into spot-level slices, respectively. The category of each spot is determined by the cell-type that predominantly occupied that spot. Evident batch effects between simulated slices can be observed. b The overview table of metrics calculated in scenario 1 demonstrates the superior performance of INSTINCT. INSTINCT reaches the highest score in 9 of the 12 metrics, and ranks second in 2 metrics. It also ranks first on the group-specific overall scores of clustering performance and representation quality, as well as the final overall score, and ranks second on the overall score of batch correction. c The level of confusion among spot-types increases continuously and batch effects become more complex across the first three scenarios. INSTINCT consistently outperforms the benchmarking methods. d Two different transformations are applied to the data of scenario 1 to generate scenario 4 and 5, in order to simulate scenarios where the proportions of spot-types differ between slices. INSTINCT results in satisfying integration outcomes. e Based on scenario 1, scenario 6 is generated to test the ability of INSTINCT to preserve rare spot-types. INSTINCT succeed in preserving the rare spot-type which mainly exists in the first slice. Source data are provided as a Source Data file.