NCCL Debugging Guide: Ring, Tree, CollNet Algorithms
NCCL debugging guide for distributed training: set NCCL_DEBUG, tune NCCL_ALGO for Ring, Tree, CollNet, and fix GPU network failures.
AI Engineering
Nvidia Blackwell: Microscaling, FP4, and FP6 Formats
Nvidia Blackwell microscaling and the new FP4 formats double inference speeds. Dive into how the second-generation Transformer Engine uses scale factors and sparsity for AI workloads.