smallvis

Documentation related to the smallvis R package:

Installing

install.packages("devtools")
devtools::install_github("jlmelville/smallvis/smallvis")
library(smallvis)

Prerequisites

• The vizier package to plot the coordinates during optimization. It’s not on CRAN, and therefore requires a fairly new version of devtools (1.9 or greater) to install this as a dependency from github.
• Similarly, the mize package (which is on CRAN but not in a sufficiently advanced state currently).
• Optionally, RSpectra package, which is used only if you want to initialize from a spectral method (Y_init = "laplacian" or Y_init = "normlaplacian"). If not present, then the standard R function eigen is used, but it’s a lot slower.

Explorations of smallvis

I have written some longer form documents that demonstrate the output of smallvis and its various options, with an emphasis on t-SNE.

• The datasets used in my ruminations.
• Some theory showing a comparison of cost functions and gradients.
• Some material on various spectral methods, which justifies the use of Laplacian Eigenmaps for initialization (a bit).
• A look at the effect of the scaling and PCA and whitening preprocessing options on t-SNE.
• How perplexity affects t-SNE.
• Using intrinsic dimensionality to automatically select perplexity. Also, non-global perplexities.
• Distance-preserving methods (geodesic as well as Euclidean).
• A comparison of various flavours of Stochastic Neighbor Embedding (t-distributed and otherwise).
• A comparison of NeRV and JSE and a look at symmetric JSE (SJSE).
• Heavy-Tailed SSNE (HSSNE).
• Optimizing the Heavy-Tail parameter in HSSNE.
• Testing some different initialization methods for t-SNE. Related: does using an average distance matrix based on the result of multiple random initializations do better than just picking the result with the lowest error? (It doesn’t.)
• A look at an exact gradient version of LargeVis.
• Some initial results using UMAP and related methods. Also, looking at the effect of borrowing some features from UMAP to see the effect on t-SNE (sadly, not much).
• t-SNE normalization.
• An evaluation of late exaggeration in t-SNE, as suggested in the FIt-SNE paper.
• Elastic Embedding and a t-Distributed variant.
• Optimizaton: L-BFGS (and qSNE).
• Optimizaton: Spectral Direction.
• Optimizaton: Conjugate Gradient.
• Optimizaton: SGD methods.
• Perplexity and three clusters from from How to Use t-SNE Effectively.
• How hard is it to unroll the Swiss Roll with SNE?
• Using input bandwidths in SNE output functions.
• Multiscale perplexities as used in multiscale t-SNE.
• The opt-SNE settings applied to smaller sized datasets.
• Global SNE (g-SNE).
• t-SNE using f-Divergences.
• ABSNE.
• Related: a bibliography of interesting (to me) t-SNE and variants.