23 docs tagged with "gpu"

EKS GPU node strategy, Karpenter·KEDA·DRA resource management, NVIDIA GPU stack, AWS Neuron stack — the accelerated computing layer covering GPUs and AWS custom accelerators

In-depth technical documentation on the architecture, deployment, and operations of the Agentic AI Platform

Covers GPU workload architecture for EKS Hybrid Nodes and DGX H200 SR-IOV and InfiniBand high-performance networking configuration.

Version-specific GPU checkpoint/restore constraints and an EKS graceful-drain and warm-start evidence procedure (Experimental)

Hands-on guide to deploying large open-source models on EKS, based on the GLM-5.1 experience

Optimal node strategies for GPU workloads across EKS Auto Mode, Karpenter, MNG, and Hybrid Nodes

Guide to building Agentic AI platform using Amazon EKS and open-source ecosystem

2-Tier GPU autoscaling (KEDA·Karpenter), DRA compatibility, and operational lessons learned from large MoE model (GLM-5·Kimi K2.5) deployments for LLM serving

GPU resource management and cost optimization using Karpenter, KEDA, and DRA on EKS

Designs a highly available GenAI inference layer for EKS Hybrid Nodes GPU nodes with resource isolation (taints), a hybrid-only NVIDIA Device Plugin deployment, and a Karpenter-based cloud GPU fallback NodePool.

Debugging guide for GPU/AI workloads on EKS

A hands-on guide to using on-premises GPU nodes as the primary inference tier on EKS Hybrid Nodes, and resolving DGX H200 SR-IOV VF name inconsistency through driver compatibility, persistent naming, and systemd orchestration

EKS architecture overview for maximizing LLM Inference performance — starting point for vLLM, KV Cache-Aware Routing, Disaggregated Serving, LWS multi-node, and GPU autoscaling

A framework guide covering the DRA core model (DeviceClass, ResourceClaim, ResourceSlice), resource types beyond GPUs (NICs, interconnects, FPGAs), and adoption criteria

Official GPU, Trainium2 and Inferentia2 specifications, Llama 4 model requirements, and a plan for measuring performance and cost

llm-d architecture concepts, KV Cache-aware routing, Disaggregated Serving, EKS Auto Mode integration strategy

A guide to the GPU infrastructure, inference framework, and inference optimization layers, with a single map of the end-to-end LLM inference request path and per-layer tuning levers — inference gateway, prefill/decode disaggregation, KV cache-aware routing, LMCache, and cache-hit strategy.

Architecture concepts, distributed deployment strategies, and performance optimization principles for Mixture of Experts models

Hardware requirements, four workload families, and a measurement plan for comparing aggregated and disaggregated NVIDIA Dynamo serving on EKS

Architecture and EKS integration for GPU Operator, DCGM, MIG, Time-Slicing, and Dynamo

Production deployment and configuration reference architecture for the Agentic AI Platform

5 key challenges faced when operating Agentic AI workloads

vLLM PagedAttention, parallelization strategies, Multi-LoRA, and hardware support architecture