What is a GPU-hour?
Written by Arnon Shimoni
✓ Expert
Last updated on:
What is a GPU-hour?
A GPU-hour is one GPU allocated to a customer for one hour, and it's the standard billing unit of GPU cloud computing. A customer running 8 GPUs for 3 hours consumes 24 GPU-hours. Rates are set per GPU model (an H100-hour and a B200-hour are different products), and the meter typically bills allocation: the time the GPU was held for you, whatever you ran on it.
That allocation principle is the single most misunderstood thing about the unit, so it goes first: idle time on a reserved GPU is billable, because the capacity was yours and unavailable to anyone else. Utilization is the customer's optimization problem, and the provider's meter records provisioned time and state, per instance, per tenant (the mechanics live under neocloud metering).
What does a GPU-hour cost?
It depends on the GPU, the commitment, and the provider, and the spread is wide:
Arrangement | Typical shape |
|---|---|
On-demand | Public per-hour rate, billed in arrears. H100 rates have ranged from roughly $2 to $10+ per hour across providers |
Reserved / committed | Discounted rate for capacity commitments over a term, the bulk of enterprise spend |
Spot / preemptible | Deep discount for revocable capacity |
Per-minute retail | The no-commitment end: e.g., Hot Aisle rents AMD GPUs at a flat $1.99 per GPU-hour, billed by the minute, no contract |
The per-minute end matters more than its revenue suggests: it's how developers evaluate hardware (one GPU, one minute, credit card) before anyone signs a commitment, and it's how challenger silicon like AMD gets tried at all. Billing granularity is a go-to-market decision wearing an engineering costume.
A worked example: a team fine-tuning on 16 GPUs for 60 hours at a $2.50 blended rate consumes 960 GPU-hours and pays $2,400. The same job on badly utilized reserved capacity of 64 GPUs costs 4x that, which is why the allocation-vs-consumption distinction decides real money.
Why bill allocation instead of work done?
Because reserved capacity has opportunity cost. A GPU held for one customer can't serve another, so consumption-based billing on reserved capacity would return the utilization risk to the provider, who controls neither the workload nor the idle time. The market's answer is layered instead: allocation billing on reserved capacity, metered billing for on-demand burst, spot for whatever's left. The commercial structures on top (minimum commits, drawdowns, overage charges) are covered under GPUaaS billing.
Where does the GPU-hour break down as a unit?
Three places. Fractional workloads: small inference jobs need a slice of a GPU, so MIG-based fractional units subdivide the hour. Orchestrated services: when the product is a managed SLURM cluster or serverless pods, providers meter profiles (the service SKU) rather than raw silicon, because that's what the customer bought. And hosted inference: once the provider serves models rather than machines, the unit becomes the token, and the GPU-hour drops out of the customer's view entirely (see token factory).
The unit also can't express quality. Two providers' H100-hours differ in interconnect, storage locality, and scheduler behavior, which is part of why the price spread is so wide... the GPU-hour prices the silicon, and the silicon was never the whole product.
Comparing GPU-hour rates across generations has a trap in it, too: each new GPU generation does more work per hour, so a higher hourly rate can be a lower cost per unit of work. The honest comparison is cost per training run or per million tokens served, which requires the meter and the price book to live in one system. That's the ledger argument, and Solvimon's Meter exists to make it.
FAQ
Is a GPU-hour the same as a compute hour?
No. A compute hour usually means a whole instance (CPU, memory, possibly several GPUs). GPU-hours count each GPU separately: one 8-GPU node for one hour is 8 GPU-hours.
Are GPU-hours billed per second or per hour?
Granularity varies by provider: per-minute and per-second metering exist, per-hour rounding still does too. At scale the rounding policy is real money, and it's specified in the contract, or should be.
What's a node-hour?
A whole configured machine (typically 8 GPUs plus CPU, memory, networking) for one hour. Providers selling by node-hour bundle the configuration; per-GPU-hour pricing unbundles it.
Do prices per GPU-hour fall over time?
Sort of! Per hour, unevenly, but per unit of work, yes: newer generations deliver more per hour. Provider pricing pages reprice with every generation, which is one reason the neocloud billing price book has to be versioned.
Related
GPUaaS billing: the commercial structures on top
Neocloud metering: how the hours get measured
Token factory: where the unit stops being the hour
Usage-based pricing: the general model
Ready for billing v2?
Solvimon is monetization infrastructure for companies that have outgrown billing v1. One system, entire lifecycle, built by the team that did this at Adyen.
