Accelerated Computing Infrastructure
3 min read
This section covers three infrastructure decisions on Kubernetes: which accelerated instances to use, how to schedule workloads on them, and how to manage drivers and accelerator partitioning. It includes NVIDIA GPUs and AWS accelerators such as Trainium and Inferentia. Inference frameworks such as vLLM and llm-d depend on this infrastructure.
EKS GPU Node StrategyAuto Mode vs Karpenter vs Managed Node Group vs Hybrid Node — optimal node selection by workload, security hardening, troubleshooting.GPU Resource ManagementKarpenter NodePool, KEDA scaling, DRA dynamic resource allocation, Spot/Consolidation cost optimization strategies.NVIDIA GPU StackGPU Operator ClusterPolicy, DCGM monitoring, MIG·Time-Slicing partitioning, Dynamo inference framework.AWS Neuron StackTrainium2/Inferentia2, Neuron SDK 2.x, aws-neuron-device-plugin, NxD Inference, vLLM Neuron backend.CRIU GPU Migration (Preview)Technology trends for GPU workload checkpoint/restore on Spot reclaim and node replacement, plus a validation checklist for EKS adoption scenarios.
Selection Guide
If focused on NVIDIA, read Node Strategy → Resource Management → NVIDIA Stack; if considering AWS silicon (Trainium/Inferentia), read Node Strategy → Neuron Stack.
Related Documents
- EKS Debugging — GPU and AI Workloads — Troubleshooting playbook for GPU nodes, drivers, and DCGM
- vLLM Model Serving — The inference engine layer above this one
- llm-d EKS Auto Mode — Disaggregated Serving and Auto Mode case study