KV Cache Optimization (vLLM Deep Dive + Cache-Aware Routing)
Summary of core technologies like vLLM PagedAttention, Continuous Batching, FP8 KV Cache, and comparison of llm-d/NVIDIA Dynamo KV Cache-Aware Routing and Gateway configuration
3 docs tagged with "kv-cache"
View all tagsllm-d Based EKS Distributed Inference Guide
llm-d architecture concepts, KV Cache-aware routing, Disaggregated Serving, EKS Auto Mode integration strategy
NVIDIA Dynamo Inference Benchmark
Benchmark comparing Aggregated vs Disaggregated LLM serving performance using NVIDIA Dynamo — Running AIPerf 4 modes in an EKS environment