Cloudflare announced in June 2026 that its AI Gateway product has processed more than one trillion tokens since launching in September 2023. For context, that is roughly the equivalent of 750 billion words — more text than exists in the entire English-language Wikipedia, repeated several hundred times.
AI Gateway started as a thin proxy layer that sat between an application and AI providers like OpenAI, Anthropic, or Hugging Face. The value proposition was simple: one API endpoint to rule all your model calls, with logging and basic rate limiting included. The product has since expanded considerably.
The current feature set includes semantic caching — Cloudflare's infrastructure checks whether an incoming prompt is semantically similar to a previously answered one and returns the cached response if it is, without forwarding the request to the upstream provider. The company says this reduces API costs by an average of 35 percent for customers with repetitive workloads like customer support chatbots or document classification pipelines.
Rate limiting can now be configured per provider, per model, per user ID, or per custom header value. A startup running a freemium AI product can, for example, cap free-tier users at 50 requests per hour to gpt-4o-mini while letting paid users pass through uncapped to claude-3-5-sonnet — all enforced at Cloudflare's edge before a single request reaches the provider.
Observability features added in Q1 2026 include token-level cost attribution, error rate dashboards broken out by model and geographic region, and webhook alerts for provider outages. The latency P99 histogram per provider has become a practical tool for companies running multi-provider routing strategies — send the request to whichever provider is fastest right now.
The product is currently free up to 100,000 requests per day. Beyond that, pricing is $0.50 per million requests. For companies already running their infrastructure on Cloudflare Workers, the integration is essentially zero-configuration.