NVIDIA has introduced multi-GPU support for Uniform Manifold Approximation and Projection (UMAP) within its cuML and cuVS 25.06 libraries. This enhancement targets the computationally intensive all-neighbors kNN graph construction, a critical step in the UMAP algorithm. The new capability allows for end-to-end scaling, significantly reducing training runtimes for datasets containing tens to hundreds of millions of vectors.
Key Points
- NVIDIA cuML and NVIDIA cuVS 25.06 introduced multi-GPU support for UMAP's all-neighbors kNN graph construction.
- This feature enables end-to-end scaling and reduces training runtimes for datasets with tens to hundreds of millions of vectors.
- The multi-GPU implementation partitions datasets into balanced clusters, computes local kNN graphs independently, and merges them into a global graph.
- This approach avoids all-to-all communication, preserving embedding quality at massive scales.
- Empirical results showed up to a 74x speedup over projected CPU runtimes using eight NVIDIA H100 GPUs.
- The method makes UMAP feasible for multi-hundred-gigabyte workloads, executing them in minutes.
- Two hyperparameters,
knn_n_clustersandknn_overlap_factor, control the trade-off between space, time, and quality.
Context
UMAP is a dimensionality reduction technique used for visualization and feature extraction in applications like exploratory data analysis, topic modeling, and single-cell analysis. According to the NVIDIA Developer Blog, iterative and exploratory workflows require UMAP to run repeatedly, and the cost increases substantially with larger datasets. The all-neighbors kNN graph construction is a critical and expensive step, especially as datasets scale.
Why It Matters
This update addresses a key bottleneck in UMAP's scalability, allowing researchers and data scientists to process significantly larger datasets more quickly. The ability to run UMAP on multi-hundred-gigabyte workloads in minutes, rather than hours or days, can accelerate iterative analysis and exploration, potentially leading to faster insights and model development.
What To Do
- Review the NVIDIA cuML and NVIDIA cuVS 25.06 documentation for details on multi-GPU UMAP implementation.
- Experiment with the
knn_n_clustersandknn_overlap_factorhyperparameters to optimize for specific dataset sizes and quality requirements. - Compare the performance of multi-GPU UMAP against single-GPU or CPU-based implementations for large-scale dimensionality reduction tasks.
- Explore the cuVS all-neighbors API for applications requiring standalone all-neighbors graph construction.
