Benchmarking FP8 Stability: Where Gradients Go to Die
FP8 is the new frontier for training efficiency, but it breaks in the most sensitive layers. We dissect the E4M3/E5M2 split and how to spot divergence.
FP8 is the new frontier for training efficiency, but it breaks in the most sensitive layers. We dissect the E4M3/E5M2 split and how to spot divergence.