RadixAttention Prefix Caching: Radix Trees vs PagedAttention
RadixAttention in SGLang uses radix trees for KV cache optimization. Compare vs vLLM PagedAttention for multi-turn and agent workloads.
Tag: LLM
· StrategySqueezing the Inference Lever: The Economics of LLM Throughput
Inference price isn't a fixed cost-it's an engineering variable. We break down the three distinct levers of efficiency: Model Compression, Runtime Optimization, and Deployment Strategy.
LLMs are Terrible Backends (Unless You Force JSON)
Non-determinism is a bug, not a feature. We explore how to whip the model into compliance using Enforcers, Pydantic, and Constrained Generation.