Part 2 of many example images of PaCMAP. A continuation of Notes on PaCMAP and PaCMAP Examples.
Here I have tried to make as direct a comparison with UMAP as I can. I again fiddled with the PaCMAP code to turn off some things: non-PCA data is centered but not range-scaled and I changed the PCA initialization in PaCMAP to scale the standard deviation to 1 for all columns. Back in PaCMAP Examples I speculated that an initialization like this would avoid the poor results I saw when I turned off mid-pair weights.
For UMAP, I used uwot which I have easy control over to mess about with for my needs. I used the exact nearest neighbors for the UMAP case because I have pregenerated them for these datasets which removes a large source of computational time. For initialization, I used the coordinates from PaCMAP at iteration 0.
In the table below, the first row is PaCMAP. The first column uses mid-near weights as usual, the second doesn’t use them and sets the near-pairs to 1 for all iterations. This is to test my theory that scaling the PCA initialization should be good enough for PaCMAP. It also removes the influence of the mid-near pairs vs UMAP. Additionally, if it works well enough without mid-near interactions, that would suggest you could implement something PaCMAP-like fairly easily in existing UMAP-style code bases (like uwot).
The second row is UMAP results. The left column is UMAP with default settings. The right column is t-UMAP, which should be “gentler” in terms of its forces (although not as gentle as PaCMAP).
The third row is UMAP results but I use the 100D PCA input like PaCMAP would (if the input dimensionality > 100), and also the near pairs are the scaled nearest neighbors. Previous results suggest that this won’t have a noticeable effect except on macosko2015 and ng20. Again UMAP is on the left and t-UMAP is on the right.
As we don’t have enough images as it is, I have added two other datasets: coil100, which is like coil20 but has more images; and isofaces, a more manifold-like dataset, beloved of many a spectral-based embedding paper.
Finally, sorry for not having useful titles on the images.
UMAP, t-UMAP and PaCMAP
This the one dataset where the dcorr is noticeably worse for the PaCMAP result without the mid-pair forces vs with it.
