Inference Gateway
Published 2026-04-20Updated 2026-06-262 min read
Overviewโ
The core data plane of Agentic AI platforms is the inference gateway. A 2-Tier architecture is recommended: kgateway (Tier 1) handles authentication, rate limiting, and guardrails, while Bifrost (Tier 2) performs model routing, fallback, and cost tracking. This section provides routing strategy overviews, production deployment guides, Cascade Routing tuning, and OpenClaw implementation examples.
Document Listโ
๐๏ธ Inference Gateway Deployment
Step-by-step deployment guide for kgateway-based Inference Gateway (basic/advanced/troubleshooting)
๐๏ธ 1. Basic Deployment
kgateway installation, HTTPRoute configuration, Bifrost Gateway Mode setup
๐๏ธ 2. Advanced Features
LLM Classifier, CloudFront/WAF, Semantic Caching configuration
๐๏ธ 3. Troubleshooting
Common issues and solutions during Inference Gateway deployment and operations