Through 2023 and into 2024, the NVIDIA H100 GPU was the most difficult piece of hardware to acquire in the technology industry. Waitlists stretched to six months or longer. Cloud providers rationed GPU hours. Startups built company strategies around access to compute, not the applications themselves. That constraint has materially eased in 2025 and 2026 — but a different shortage has emerged to replace it.
The new bottleneck is power and cooling. A single rack of NVIDIA H100 GPUs consumes between 60 and 80 kilowatts. A facility deploying 10,000 H100s — a size that a mid-tier AI research organization might require — needs roughly 800 megawatts of electrical capacity, plus the cooling infrastructure to match. Data center construction pipelines that take two to three years to complete are struggling to keep pace with the pace at which hyperscalers are ordering hardware.
Microsoft, Google, Amazon, and Meta have all announced multi-billion dollar data center investment programs. Microsoft alone has committed over 80 billion dollars in infrastructure investment through 2025. The constraint is no longer the chips — it is the land, the power purchase agreements, the utility grid connections, and the construction labor to build the facilities.
NVIDIA's Blackwell architecture, which began shipping in volume in 2025, introduces the GB200 NVL72 system — a rack-scale product that connects 36 Grace CPUs and 72 Blackwell GPUs with NVLink, delivering what NVIDIA describes as 30 times the performance of H100 for LLM inference workloads. The power draw per rack increases accordingly, deepening the infrastructure challenge.
AMD's MI300X has made real inroads in inference workloads where its larger HBM memory capacity reduces the need to swap model weights. Microsoft Azure, Meta, and several cloud providers have deployed MI300X at meaningful scale. The competitive dynamic between NVIDIA and AMD is now real, even if NVIDIA's software ecosystem remains a significant advantage for training workloads.
For organizations planning AI infrastructure, the practical implication is that reserved capacity planning has replaced spot market purchasing as the dominant strategy. Long-term contracts with cloud providers, co-location agreements with power-rich facilities, and partnerships with hyperscalers are replacing the assumption that compute is available on demand.
The GPU shortage taught the industry that chips could be the scarce resource. The next phase teaches a different lesson: the electrons that power the chips are the constraint that no amount of engineering can compress on a short timeline.