HBM-Aware Load Balancing with libtpu and GKE
CPU load is a trailing indicator for AI inference. Discover how to use libtpu metrics and the GKE Gateway API to build high-density, memory-aware traffic routing for TPUs.
AI Infrastructure
Beyond Vibe-Checks: Trajectory Evaluation & Synthetic Adversaries
Is your agent actually reasoning, or just lucky? Discover why trajectory analysis and synthetic red-teaming are the only ways to build production-grade autonomous systems.
Stateful Agents on K8s: Redis is Your Bottleneck, Not the Vector DB
Agents are stateless. Their memory is not. Scaling the LLM reasoning loop is trivial compared to solving the transactional concurrency of agent memory on Kubernetes.
Writing Pallas Kernels for JAX: Stepping Outside the XLA Safety Net
When XLA's heuristics fail for custom attention mechanisms, you can't just hope for a compiler update. Here is how you write Triton-like kernels directly in Python using JAX Pallas.
Speculative Decoding: Cheating Physics for Latency
Using a 'Draft' model costs 10% more VRAM but saves 50% Latency. Here is the mechanics of the gamble.
My Profiling Nightmare: The Warp Stall
A war story of chasing a 5ms latency spike to a single loose thread. How to read Nsight Systems and spot Warp Divergence.