NVIDIA has introduced NVLink Fusion and NVHBM to support the increasing demands of AI factories for larger models and more complex reasoning workloads. These technologies are designed to help hyperscalers and AI-native companies deploy custom AI accelerators, or XPUs, into the NVIDIA AI infrastructure platform.
Key Points
- NVLink Fusion and NVHBM together deliver up to 30% higher memory bandwidth compared to standard HBM4e.
- The combined technologies provide 25% more compute die area.
- They achieve 15% lower HBM power usage compared to standard HBM4e.
- This results in a 30% overall end-to-end XPU performance increase.
- NVHBM integrates a custom base die and PHY, moving the memory controller into the 3D HBM stack.
- NVLink Fusion uses a chiplet architecture and sixth-generation NVLink fabric to connect custom XPUs, CPUs, and GPUs at rack scale.
- NVHBM reduces PHY and support area by up to 67% compared to the JEDEC HBM4e standard.
Context
According to the NVIDIA Developer Blog, AI factories require high-bandwidth memory, sufficient package and silicon area, efficient power delivery, and a resilient supply chain to deploy custom accelerators at scale. NVLink Fusion provides the connective technology and IP for integrating custom XPUs and CPUs into the NVIDIA AI infrastructure platform, leveraging the NVIDIA scale-up and scale-out technology stack, ecosystem, and MGX rack-scale architecture. NVHBM complements this by offering a custom HBM base-die technology, designed and validated with memory vendors, to increase memory bandwidth, save area, and lower power consumption.
Why It Matters
These advancements allow builders of custom AI silicon to optimize their accelerators for various workloads, such as inference serving, recommendation systems, multimodal pipelines, or internal training. The increased memory bandwidth and compute area, coupled with reduced power consumption, can lead to more efficient and powerful AI systems.
What To Do
- Note the performance improvements, including 30% higher memory bandwidth and 25% more compute die area, when evaluating custom XPU designs.
- Consider the implications of 15% lower HBM power usage for data center energy efficiency.
- Examine how the integration of NVLink Fusion and NVHBM might simplify the deployment of custom XPUs and CPUs within the NVIDIA AI infrastructure.
- Watch for further details on how NVHBM's area savings, achieved by moving the memory controller into the 3D HBM stack, can free up space for workload-specific optimizations.