AI Gateway

AI Gateway provides centralized management for LLM traffic, offering security, observability, and advanced routing through integration with inference extensions Inference Extension Project Mechanisms to integrate inference/AI processing into gateway traffic flows. .

Explore what you can do with AI Gateway in the following guides.

About AI Gateway Set up AI Gateway LLM cloud providers Ollama for local LLMs Authenticate LLM providers Model failover Function calling Prompt enrichment Prompt guards AI Gateway metrics Cleanup Inference Extension for local LLMs