Skip to main content

East-West Traffic Optimization: Balancing Performance and Cost

Published 2026-02-04Updated 2026-06-305 min read

Overview​

This guide covers optimizing inter-service communication (East-West traffic) in Amazon EKS from latency minimization and cost efficiency perspectives. It progressively addresses scenarios from single cluster to multi-AZ, multi-cluster, and multi-account environments.

When East-West hops increase from 1 to 2, p99 latency grows by milliseconds. Cross-AZ traffic incurs AWS bandwidth charges ($0.01/GB). This guide analyzes layer-by-layer options from Kubernetes-native features (Topology Aware Routing, InternalTrafficPolicy) to Cilium ClusterMesh, AWS VPC Lattice, and Istio service mesh, with quantitative latency, overhead, and cost comparisons.

Key Benefits​

ItemImprovement
Network LatencySame-AZ routing via Topology Aware Routing, sub-ms p99
Cost Savings~$100/month at 10TB/month by eliminating cross-AZ traffic
Operational SimplicityClusterIP-based inter-service optimization without LBs
DNS PerformanceNodeLocal DNSCache: ms → sub-ms DNS lookup
ScalabilityConsistent extension path to multi-cluster/account
🔀 Kubernetes Service Type Comparison
Service type selection guide for East-West traffic
ClusterIPL4Recommended
In-cluster virtual IP, kube-proxy NAT (iptables=random, IPVS=round-robin)
+ Simple, auto DNS, no extra cost
- Random cross-AZ, NAT overhead
HeadlessL4
No clusterIP, DNS exposes all Pod IPs directly, no proxy
+ Direct connection (min latency), gRPC DNS round-robin, required for StatefulSet
- Client LB logic needed, DNS refresh delay
Internal NLBL4
AWS NLB Controller, L4 operation, Instance/IP mode
+ Multi-AZ HA, ultra-low L4 latency, static IP
- NLB hourly cost, Instance mode cross-AZ cost increase
Internal ALBL7
AWS ALB Controller, L7 operation, IP mode only
+ L7 features (path routing, WAF, gRPC), free cross-zone
- ALB hourly + LCU cost, ms-level added latency
💡 Selection Guide: Default: ClusterIP + TAR | StatefulSet: Headless | L7 needed: Internal ALB (IP mode) | L4 external: Internal NLB (IP mode)

Implementation​

  1. Topology Aware Routing — service.kubernetes.io/topology-mode: Auto
  2. InternalTrafficPolicy Local — Same-node-only traffic
  3. Pod Topology Spread Constraints — AZ-balanced distribution
  4. NodeLocal DNSCache — Local DNS caching per node
  5. Internal LB IP Mode — Direct Pod IP targeting
  6. Istio Service Mesh (optional) — mTLS, locality-aware routing
Choosing a Mesh Solution

Which service mesh to adopt (Istio, Cilium, Linkerd, VPC Lattice) is covered in the Service Mesh Comparison Guide. This document focuses on post-adoption latency and cost optimization.

Multi-Cluster Connection Strategies​

  • Cilium ClusterMesh: Lowest latency, Pod-to-Pod direct, eBPF-based
  • AWS VPC Lattice: Managed proxy, IAM auth, multi-account
  • Istio Multi-cluster: Full mesh features across clusters, mTLS
  • Route53 + ExternalDNS: Simplest, DNS-based
⚡ Latency & Cost Comparison by Option
Quantitative performance & cost comparison by traffic path
Option
Latency / Cost
Latency
Cost
ClusterIP
kube-proxy NAT (µs~ms). Minimal overhead vs direct
No extra cost. $0.01/GB for cross-AZ
Headless
No proxy, near-zero added latency. Only DNS lookup time
No extra cost. Same cross-AZ cost
Internal NLB (Instance)
Few ms + NodePort hop + possible kube-proxy hop
NLB hourly + data GB + increased cross-AZ cost
Internal NLB (IP)
Few ms. Direct to Pod, no NodePort hop
NLB cost, savings from avoiding cross-AZ traffic
Internal ALB
ms to tens of ms (L7 rule processing, proportional to request size/rules)
ALB hourly ($0.0225/h) + LCU. Free cross-zone
Istio Sidecar
~5ms added (client+server 2x proxy)
Open source, 0.2 vCPU/60MB per proxy per 1000rps
Cilium ClusterMesh
Pod→Pod direct. VXLAN encap overhead tens of µs
No AWS service cost. Data transfer only
VPC Lattice
Managed proxy, few ms per HTTP request
$0.025/h + $0.025/GB + $0.1/1M req
💰 10 TB/Month East-West Traffic Cost Simulation
Same-region 3-AZ EKS cluster, 10 TB (10,240 GB) inter-service traffic
InternalTrafficPolicy Local$0
Node-local traffic, zero cross-AZ
ClusterIP + Topology Hints~$30
cross-AZ reduced to ~30%
ClusterIP (default, no AZ awareness)~$68
cross-AZ ~66% (3-AZ even distribution)
Via Internal ALB~$98
ALB hourly + LCU + cross-AZ
VPC Lattice$400+
Per service hourly + per GB + per request
🎯 Scenario-Based Recommendation Matrix
Recommended solutions by service characteristics, security requirements, and operational complexity
🔧Simple internal microservices (HTTP/gRPC, latency-sensitive)Single Cluster
ClusterIP + TAR + NodeLocal DNSCache. Optionally InternalTrafficPolicy(Local) for same-node optimization
🗄️StatefulSet (DB, TCP, session required)Single Cluster
Headless service + client DNS round-robin. Schedule leader-follower in same AZ as client
🌐High-volume L7 traffic (routing/WAF needed)Single Cluster
Internal ALB (IP mode) + ClusterIP service. ALB only where needed
🔒Security-sensitive (mTLS/Zero-Trust required)Single/Multi Cluster
Istio service mesh. mTLS + AuthorizationPolicy. Review latency requirements and traffic volume
💰Multi-AZ cost optimization (high traffic, no mesh)Single Cluster
Topology Hints + IP mode LB + Pod Spread + NAT GW per AZ
🔗Multi-cluster (same account, low ops complexity)Multi-cluster
Cilium ClusterMesh. Direct Pod communication, low latency with no extra cost
🏢Multi-account/org (IAM control, small ops team)Multi-account
AWS VPC Lattice. IAM Policy-based access control, unified monitoring
🔄Simple DR cluster (low traffic)DR
DNS + Internal NLB (Route53 + ExternalDNS). DNS switching for DR

References​

  1. AWS ELB Pricing
  2. AWS Data Transfer Pricing
  3. Kubernetes Topology Aware Routing
  4. Cilium ClusterMesh
  5. AWS VPC Lattice Pricing