Inference Infrastructure Overview and Tuning Layers
A single map of the end-to-end LLM inference request path at the infrastructure layer, with tuning levers organized by layer — inference gateway, prefill/decode disaggregation, context/KV cache-aware routing, LMCache, and cache-hit strategy.