Cloudflare announced in July 2026 that its AI Gateway product had processed two trillion API requests since launch — a milestone that doubles the one-trillion mark it hit just five months earlier. The growth rate reflects how quickly AI inference has moved from experiment to production dependency in enterprise infrastructure.
AI Gateway sits between applications and upstream LLM providers. It handles retries, rate limiting, semantic caching (where similar prompts share cached completions), cost controls, and observability. The value proposition is straightforward: centralise all LLM traffic through one layer so you can monitor, control, and reduce costs without changing application code.
The request composition data Cloudflare has shared publicly is revealing. OpenAI's API accounts for approximately 58 percent of traffic by request count, down from 71 percent at the one-trillion-request milestone. Anthropic has grown from 11 percent to 19 percent of traffic in the same period. Google's Gemini API represents about 14 percent, and the remaining 9 percent is distributed across open-source model endpoints — primarily Hugging Face Inference API and self-hosted models accessed via Workers AI.
The semantic cache hit rate has reached 23 percent across the network. That means nearly one in four requests is served from a cached response, with no upstream API call and no token cost. For enterprise customers with repetitive query patterns — customer support bots, document classification, structured extraction — cache hit rates above 40 percent are common. At typical GPT-4o pricing, a 23 percent cache hit rate across two trillion requests represents billions of dollars in avoided cost.
The fastest-growing segment is agentic workflows — multi-step requests where one LLM call triggers others. Cloudflare reports these account for 31 percent of traffic by token volume, up from 8 percent twelve months ago. These chains are harder to cache but more sensitive to latency, which is where Cloudflare's edge network (now at 330 cities) provides measurable value over sending every request to a single regional endpoint.
What this data tells us: OpenAI's dominance is real but eroding at the enterprise layer. Anthropic's Claude is winning long-context and reasoning workloads. And the infrastructure abstraction layer — the "LLM router" category that Cloudflare, LiteLLM, and BrainTrust all occupy — has become load-bearing for production AI systems.