1000x Increase in AI Demand

NVIDIA announced earnings yesterday. In addition to continued exceptional growth, the most interesting observations revolve around a shift from simple one-shot AI to reasoning.

Reasoning improves accuracy for robots - like telling a person to stop and think about an answer before they reply. Here’s an example where I asked Gemini to create a financial projection for NVIDIA for the next five years.

Reasoning is compute-intensive, requires hundreds to thousands more – thousands of times more tokens per task than previous one-shot inference.

Software engineers also use reasoning extensively as AI coding agents examine code bases, plan modifications, and execute them. Each time I watch one of these reasoning traces I wonder how many GPUs are firing to produce the result.

OpenAI, Microsoft and Google are seeing a step-function leap in token generation. Microsoft processed over 100 trillion tokens in Q1, a fivefold increase on a year-over-year basis.

In addition to increased demand and greater usage, these reasoning models are driving significant volume increases in tokens as we saw in the Microsoft earnings announcement a few weeks ago.

On average, major hyperscalers are each deploying nearly 1,000 NVL72 racks or 72,000 Blackwell GPUs per week and are on track to further ramp output this quarter. Microsoft, for example, has already deployed tens of thousands of Blackwell GPUs and is expected to ramp to hundreds of thousands of GB200s with OpenAI as one of its key customers.

72,000 GPUs deployed per week is quite a statistic!

The pace and scale of AI factory deployments are accelerating with nearly 100 NVIDIA-powered AI factories in flight this quarter, a twofold increase year-over-year, with the average number of GPUs powering each factory also doubling in the same period.

To match the demand, hyperscalers are deploying more than $300b in capex this year to fund data centers, which interestingly, NVIDIA calls AI factories. What is the marketing rationale behind this framing? A new industrial revolution?

To date, the algorithmic improvements that reduce the overall model sizes are helping to staunch some of the geometric explosion in demand for AI, but it’s clear that both the demand for AI and more sophisticated reasoning are outpacing those advances.