Tag: GPUs

The infrastructure hacks required to make scale-to-zero LLM inference viable for production latency.

Don't lock into one vendor. Learn how to use an abstraction layer to route training and inference workloads to the cheapest available capacity across hyperscalers and neoclouds.

Google Cloud’s G4 architecture delivers 168% higher throughput by maximizing PCIe Gen 5 performance. This deep dive examines the engineering stack driving these gains, from direct P2P communication and NUMA optimizations to Titanium offloads. Explore how G4 transforms standard connectivity into a high-speed fabric for demanding AI inference and training.

Understanding how to partition a single GPU into multiple isolated instances for cost-efficient AI workloads, with a deep dive into NVIDIA's MIG technology and the architectural differences between GKE and EKS..