Fig. 3: LOO loss landscape reveals the origins of two distortion patterns.
From: Assessing and improving reliability of neighbor embedding methods: a map-continuity perspective
a We illustrate two discontinuity patterns on simulated Gaussian mixture data. OI discontinuity: t-SNE embeds points into well-separated clusters and creates visual overconfidence. FI discontinuity: t-SNE with an inappropriate perplexity creates many artificial fractures. b Origin of OI discontinuity: LOO loss contour plot shows distantly separated minima. We add a new input point x at one of the 4 interpolated locations x = tc1 + (1 − t)c2 where t ∈ {0, 0.47, 0.48, 1} and then visualize the landscape of the LOO loss L(y; x) using contour plots in the space of y. The middle two plots exhibit two well-separated minima (orange triangle), which cause a huge jump of the embedding point (as the minimizer of the LOO loss) under a small perturbation of x. c Origin of FI discontinuity: We show LOO loss contour plots with interpolation coefficient t ∈ {0.2, 0.4, 0.6, 0.8}. The plots show many local minima and irregular jumps. Under an inappropriate perplexity, the loss landscape is consistently fractured. Numerous local minima cause an uneven trajectory of embedding points (dashed line) when adding x at evenly interpolated locations. Source data are provided as a Source Data file.
