GPU Spot Markets and What They Signal About the AI Cycle

There's a secondary market for GPU compute that doesn't get much coverage outside of the people actively operating in it. Spot instances through the cloud providers, reserved capacity traded through brokers, bare-metal GPU clusters available through the new generation of GPU cloud companies.

The pricing in these markets is a real-time signal about supply and demand for AI compute that you can't easily read anywhere else. And for the past six months, the signal has been interesting.

The H100 Story

At peak in late 2023 through mid-2024, H100 availability was genuinely constrained. Spot rates were elevated above on-demand pricing in some markets — which is unusual and a sign of acute demand exceeding supply. Reserved capacity was selling at premiums on secondary markets. If you needed H100s and didn't have a hyperscaler commitment, you were paying significantly over rack rate and often waiting.

That picture has been changing. Spot availability has improved meaningfully. Rates in the H100 market have come down — not collapsed, but normalized back to something closer to the theoretical cost-plus-margin pricing. The spot premium that indicated acute scarcity is largely gone.

Two things drove this. First, NVIDIA accelerated Blackwell availability faster than the market expected, and large customers who were waiting on Blackwell freed up their H100 allocations. Second, some of the early hyperscaler AI build-out — particularly for the training cluster use case — has gotten ahead of immediate demand and there's more available capacity than there was six months ago.

The Blackwell Picture

B200 availability is improving but is still demonstrably constrained. The market here is less mature — less secondary trading infrastructure, fewer spot offerings — but the trend is the same directional arrow as H100 followed.

The interesting development: the new generation of GPU cloud companies built specifically around Blackwell (CoreWeave being the obvious example, but there are others) have been aggressive about locking in supply and building out the managed infrastructure around it. They're trying to solve the problem the hyperscalers solve slowly — giving you compute access without requiring you to be a Fortune 500 company with a committed cloud relationship.

The pricing from these providers is worth benchmarking against the hyperscaler equivalent. For some workloads and timelines, the unit economics favor the GPU cloud approach. For others, the hyperscaler ecosystem — compliance posture, networking integration, the managed services layer — is worth the premium. Neither is universally correct.

What the Market Signal Actually Means

When commodity markets for a resource normalize from scarcity to adequacy, a few things tend to follow.

Buyers who locked in expensive long-term contracts during the scarcity period are stuck with above-market costs until those contracts mature. Some hyperscalers signed unfavorable terms at peak GPU scarcity that are still running. This shows up as margin pressure in data center operations, even if it doesn't show up visibly in the earnings reports.

The normalization of spot markets also signals that the first phase of the AI infrastructure build-out is largely complete. Not the whole build-out — demand is still growing and capacity commitments are still running — but the acute phase where you simply couldn't get compute is over. We're entering a more normal supply-demand relationship.

The Implication for Infrastructure Timing

If you're planning significant AI infrastructure spend — either direct hardware or committed cloud capacity — the current market is different from what it was 18 months ago. The urgency that drove "sign anything to get capacity" decisions is reduced.

This doesn't mean wait. Demand is still growing, and the economics of reserving capacity at reasonable rates are still favorable relative to spot in most scenarios. But the calculus looks different when you have actual alternatives and time to evaluate them.

The spot market normalization is also a signal about the competitive moat. When a resource is scarce, the companies that have it locked up have a genuine advantage. As that scarcity eases, the advantage shifts from "we have compute" to "we do more with compute." That's a harder moat to maintain, and a more interesting one to watch.

One Thing to Watch

H100 spot rates are a lagging indicator. The leading indicator is what NVIDIA's customers are signaling about their next-cycle orders — specifically whether the Blackwell ramp is absorbing all available production or whether there's room opening up.

The secondary market will tell you what happened. The primary order book tells you what's coming. You can't see the order book. But you can watch NVIDIA's earnings commentary for capacity allocation language. That's usually where the signal is.

Price is the rearview. The order book is the windshield.

— Dustin