Cloud GPU Pricing in 2026: What It Costs to Rent an H100, B200 or RTX 5090

Renting a GPU in the cloud is the fastest way to get serious compute for deep learning: no hardware to buy, no drivers to fight with on your own machine, and access to accelerators such as the NVIDIA H100 or B200 that cost tens of thousands of dollars each. But cloud GPU pricing is confusing. The same H100 can cost $2 an hour on one platform and $12 on another, prices differ for on-demand, spot and reserved capacity, and storage, data transfer and idle time can add a lot to the bill.

This guide compares cloud GPU prices in autumn 2026, checked in late October and early November, across hyperscalers, AI-focused clouds and GPU marketplaces, explains what you actually pay for, and shows with a small Python calculator how to estimate the cost of a job and decide whether to rent or buy. Prices in this market change often, in 2026 mostly upward, so treat the numbers as a snapshot and check the provider’s pricing page before you budget.

Cloud GPU prices at a glance

The table shows typical on-demand prices per GPU-hour in the US at the end of October 2026, with examples from specific providers. Hyperscaler prices are calculated from 8-GPU instances. The low end of each range usually comes from GPU marketplaces, where prices depend on the individual host.

Older and smaller GPUs are cheaper still: an NVIDIA T4 on AWS (g4dn.xlarge) costs about $0.53 per hour, an L4 (g6.xlarge) about $0.80. They are enough for learning, small models and inference, but not for training large ones.

H100 price per hour by provider

The NVIDIA H100 is the workhorse of AI training and inference and the GPU most people compare prices on. In October 2026 the on-demand price of the same GPU differed by almost six times between providers:

Some details matter when you compare. The H100 exists in three versions: SXM (the fastest, in 8-GPU servers with NVLink), PCIe (slower, often cheaper, for example $2.89 on RunPod) and NVL. Hyperscalers rent H100s mostly as whole 8-GPU machines: an AWS p5.48xlarge costs $55.04 per hour, a Google Cloud a3-highgpu-8g $88.49, an Azure ND96isr H100 v5 $98.32. And marketplace prices like Vast.ai’s depend on the individual host and can change during the day.

Types of GPU cloud providers

• Hyperscalers: AWS, Google Cloud, Microsoft Azure, Oracle Cloud. The most expensive per GPU-hour, but they offer everything around the GPU: managed Kubernetes, storage, databases, networking, security and compliance certifications, and enterprise support. They make sense if your data and the rest of your infrastructure are already there, or if you have credits or negotiated discounts. Getting large H100 or B200 quotas often requires a request to support or a reservation.
• AI-focused clouds (neoclouds): CoreWeave, Lambda, Nebius, Crusoe, RunPod, Together AI, Hyperbolic and others. They specialize in GPUs, typically charge half as much as hyperscalers or less, and offer clusters with fast InfiniBand interconnect for multi-node training. Fewer surrounding services, but for training and inference that is often all you need.
• GPU marketplaces and community clouds: Vast.ai, RunPod Community Cloud. Independent hosts and data centers rent out their GPUs, including consumer cards like the RTX 4090 and 5090. The cheapest option, but reliability, network speed and security vary from host to host, so they suit experiments and jobs that can survive interruptions better than sensitive data.
• Notebooks with free and cheap GPUs: Google Colab, Kaggle. Colab’s free tier gives limited, unguaranteed access to a GPU for sessions of up to 12 hours; Colab Pro costs $9.99 per month for 100 compute units, which buys roughly 80 hours on a T4 or 13 hours on an A100 80GB. Kaggle notebooks offer free GPU sessions of up to 12 hours with a weekly quota, commonly reported at around 30 hours. Ideal for learning, not for long training runs.

Why cloud GPU prices went up in 2026

For several years cloud GPU prices were falling: in June 2025 AWS cut on-demand prices of its H100 instances by about 44%, and AI clouds competed to offer an H100 for under $2–3 per hour. In 2026 the trend reversed. Demand for AI compute kept growing, new Blackwell GPUs were scarce, and the global shortage of memory chips pushed up the cost of new servers.

• Nebius raised on-demand prices on October 1, 2026: H100 from $3.85 to $4.50 per hour, H200 from $4.50 to $5.40, B200 from $7.15 to $8.50, B300 from $7.85 to $9.50.
• AWS raised prices of EC2 Capacity Blocks for ML, its reserved GPU capacity, by about 15% in January 2026, by about 20% in July (P5 with H100 to $5.19 per GPU-hour, P6-B200 to $12.36) and again in October.
• RunPod and Lambda list the H100 SXM at $3.99 per hour; in spring 2026 both were quoted at $2.99 or less. RunPod’s B200 rose from about $5.89 to $7.99.

For buyers this means two things: compare prices just before you commit, and do not assume a quote from a few months ago is still valid. The same shortage made buying GPUs more expensive too, as we describe in our guide to the best GPU for AI.

On-demand, spot and reserved pricing

Most providers sell the same GPU in three ways:

Reservations are not always cheaper. Short-term guaranteed clusters with fast interconnect sell at a premium: Lambda’s 1-Click Clusters of B200s cost $8.87–9.86 per GPU-hour for terms from two weeks, more than its on-demand B200 at $6.69. You pay for the guarantee that 16 or 256 GPUs will be available at once, connected by InfiniBand.

If you use spot instances, save checkpoints often, at least every 15–30 minutes, so that an interruption costs only the work since the last save. Most training frameworks can resume from a checkpoint automatically. How epochs, batches and iterations relate, and so how often you can checkpoint, is explained in epoch vs batch size vs iteration.

Hidden costs

The GPU price per hour is only part of the bill. Before you compare providers, check:

• Storage. Datasets, checkpoints and model weights need persistent disks or network storage, billed per GB per month even while the GPU is off: on RunPod $0.05–0.20 per GB per month depending on the type, so 2 TB of checkpoints cost $100–400 a month.
• Data transfer. Hyperscalers charge for data leaving their cloud, typically around $0.05–0.12 per GB; downloading a 1 TB dataset or many model checkpoints adds up. Many AI clouds do not charge for egress.
• Idle time. A GPU that sits idle while you debug, read logs or sleep costs the same as one that trains. Stop instances when not in use and set up automatic shutdown.
• Bundled CPU, RAM and networking. Hyperscaler GPU instances include many CPU cores, a lot of RAM and fast networking, which partly explains their price; on cheaper platforms data loading can become the bottleneck.
• Multi-GPU efficiency. Eight GPUs are not eight times faster if they are connected by a slow network or your code does not scale. For multi-node training, InfiniBand or a comparable interconnect is essential.
• Minimum billing and quotas. Some providers bill by the hour or require a minimum term; hyperscalers may need days to approve a quota for large GPUs.

How to estimate the cost of a job

The script below compares the cost of the same fine-tuning job, 8 H100 GPUs for 24 hours plus 2 TB of storage, across providers. It also checks whether a more expensive B200 is cheaper per job, what spot capacity saves, and when buying a GPU pays off:

# Cloud GPU cost calculator. On-demand prices per GPU-hour, late October 2026:
# they change often, so check the provider's page before you budget.
h100 = {
    "Vast.ai (marketplace)": 2.14,
    "Lambda":                3.99,
    "RunPod (H100 SXM)":     3.99,
    "Nebius":                4.50,
    "CoreWeave":             6.16,
    "AWS (p5)":              6.88,
    "Google Cloud (A3)":    11.06,
    "Azure (ND H100 v5)":   12.29,
}

# 1. A fine-tuning job that needs 8 GPUs for 24 hours, plus 2 TB of storage for a month
gpus, hours = 8, 24
storage_tb, storage_per_gb_month = 2, 0.10
storage = storage_tb * 1000 * storage_per_gb_month
print(f"Job: {gpus} x H100 for {hours} h = {gpus * hours} GPU-hours, storage ${storage:.0f}/month")
for provider, price in h100.items():
    print(f"  {provider:22s} ${price:5.2f}/GPU-h -> ${gpus * hours * price + storage:7,.0f}")

# 2. Price per hour is not price per job: B200 costs more per hour but finishes sooner (compute only)
b200_speedup = 2.0       # roughly what MLPerf training results show for B200 vs H100
for name, h100_price, b200_price in [("Lambda", 3.99, 6.69), ("RunPod", 3.99, 7.99), ("Nebius", 4.50, 8.50)]:
    print(f"{name:7s} H100 job ${gpus * hours * h100_price:,.0f}  vs  B200 job "
          f"${gpus * hours / b200_speedup * b200_price:,.0f}")

# 3. Spot or preemptible instances: cheaper, but interruptions cost time
spot_price, overhead = 2.30, 0.20    # AWS p5 spot was about $1.95-2.50 in August 2026; assume 20% lost to restarts
print(f"Spot H100 at ${spot_price:.2f}/h with {overhead:.0%} restart overhead: "
      f"${gpus * hours * (1 + overhead) * spot_price + storage:,.0f}")

# 4. Rent or buy? Hours of use after which buying pays off
def break_even_hours(card_price, rent_per_hour, watts, kwh_price=0.15):
    electricity = watts / 1000 * kwh_price        # cost of running your own card for an hour
    return card_price / (rent_per_hour - electricity)

for card, price, rent, watts in [("RTX 5090", 4500, 1.19, 575),
                                 ("RTX 5090", 4500, 0.60, 575),
                                 ("H100 PCIe", 30000, 2.89, 350)]:
    h = break_even_hours(price, rent, watts)
    print(f"{card:9s} bought for ${price:,} vs rented at ${rent:.2f}/h: "
          f"pays off after {h:,.0f} h (~{h / 24 / 30:.0f} months of 24/7 use, {h / 8 / 22:.0f} months at 8 h/workday)")

Job: 8 x H100 for 24 h = 192 GPU-hours, storage $200/month
  Vast.ai (marketplace)  $ 2.14/GPU-h -> $    611
  Lambda                 $ 3.99/GPU-h -> $    966
  RunPod (H100 SXM)      $ 3.99/GPU-h -> $    966
  Nebius                 $ 4.50/GPU-h -> $  1,064
  CoreWeave              $ 6.16/GPU-h -> $  1,383
  AWS (p5)               $ 6.88/GPU-h -> $  1,521
  Google Cloud (A3)      $11.06/GPU-h -> $  2,324
  Azure (ND H100 v5)     $12.29/GPU-h -> $  2,560
Lambda  H100 job $766  vs  B200 job $642
RunPod  H100 job $766  vs  B200 job $767
Nebius  H100 job $864  vs  B200 job $816
Spot H100 at $2.30/h with 20% restart overhead: $730
RTX 5090  bought for $4,500 vs rented at $1.19/h: pays off after 4,077 h (~6 months of 24/7 use, 23 months at 8 h/workday)
RTX 5090  bought for $4,500 vs rented at $0.60/h: pays off after 8,759 h (~12 months of 24/7 use, 50 months at 8 h/workday)
H100 PCIe bought for $30,000 vs rented at $2.89/h: pays off after 10,573 h (~15 months of 24/7 use, 60 months at 8 h/workday)

The same 192 GPU-hours cost from $611 on a marketplace to $2,560 on Azure. AI clouds land around $1,000, AWS at about $1,500. Three more conclusions:

• Compare price per job, not price per hour. NVIDIA’s B200 trained about twice as fast as the H100 in MLPerf Training benchmarks (2× for GPT-3 pretraining, 2.2× for Llama 2 70B fine-tuning). With that speedup, a B200 on Lambda finishes the job for $642 instead of $766, and on RunPod and Nebius it costs about the same as an H100 while giving you the result twice as fast. The speedup depends on the workload, so measure it on a short run of your own job first.
• Spot is cheaper even with interruptions. With 20% of compute lost to restarts, spot H100s cost $730 instead of $1,521 on-demand on AWS.
• Storage is a fixed cost. $200 a month for 2 TB is a third of the cheapest option’s bill; delete old checkpoints.

Which GPU to rent for which task

Start from memory: the model, its activations and optimizer states must fit into GPU memory. Our guide to the best GPU for AI includes a calculator of how much VRAM you need for running and fine-tuning language models.

• Learning deep learning: Colab or Kaggle for free, then an RTX 4090/5090 or an A100 by the hour when you need more. A typical course project, such as training a convolutional neural network on a dataset of images, costs a few dollars to a few tens of dollars.
• Fine-tuning small and medium LLMs with LoRA or QLoRA (7–32B parameters): RTX 4090/5090 (24–32 GB), L40S (48 GB), A100 80GB or RTX PRO 6000 (96 GB). At $0.50–2.50 per hour these are the best value.
• Running a 70B model or serving LLMs in production: H100 or H200, often several of them. A 70B model in 8-bit needs about 74 GB, so it barely fits on an 80 GB H100, while the H200’s 141 GB leaves room for long contexts and many parallel requests.
• Full fine-tuning or pretraining large models: clusters of H100, H200 or B200 with InfiniBand, usually reserved for weeks or months.
• Image and video generation: RTX 4090/5090 or L40S for image models and LoRA training; H100 and newer for video models.

Rent or buy?

The last part of the script shows when buying pays off. An RTX 5090 bought for $4,500 (roughly its street price in August 2026, more than twice the $1,999 list price; marketplace offers later went even higher) pays for itself after about 4,000 hours of use compared to renting at RunPod’s $1.19 per hour: six months of round-the-clock use, or almost two years at eight hours per working day. Against a marketplace price of $0.60, it takes about a year of 24/7 work. An H100 PCIe at around $30,000 pays off after roughly 10,600 hours, about 15 months of continuous use, and that is before the cost of a server, cooling and maintenance.

Rough rule: buy if the GPU will be busy most of the day for a year or more and the models fit in its memory; rent if you need GPUs occasionally, need more memory or more GPUs than you can buy, or want to try the latest hardware. Many teams combine both: a local GPU for development and debugging, cloud GPUs for large training runs.

How to choose a cloud GPU provider

1. Define the job: which GPU and how much memory, how many GPUs, for how long, and whether it can survive interruptions.
2. Compare like for like: the same GPU version (SXM vs PCIe), per-GPU price, and what is included: CPU, RAM, storage, egress.
3. Check availability, not only the price list. The cheapest GPU is useless if it is never available; try to launch an instance before planning around it.
4. Run a short benchmark of your actual job on two or three candidates and compare the cost per job.
5. Consider data and security: where the data is stored, data residency requirements, whether a marketplace host is acceptable for your data.
6. Plan for interruptions and budget: checkpoints, automatic shutdown of idle instances, spending alerts.

FAQ

How much does it cost to rent an H100?

In October 2026, about $2–2.50 per hour on GPU marketplaces, $4–6 on AI-focused clouds such as Lambda, RunPod, Nebius and CoreWeave, and $7–12 on AWS, Google Cloud, Azure and Oracle. Spot or preemptible H100s cost from under $1 to about $2.50 per hour, but can be interrupted.

What is the cheapest cloud GPU?

Free notebooks on Google Colab and Kaggle give limited access to a GPU at no cost. Among paid options, consumer GPUs such as the RTX 4090 and RTX 5090 on marketplaces like Vast.ai cost from about $0.30–0.50 per hour, and older data center GPUs such as the T4 or A100 can be found for under $1. For large models, the cheapest H100s are on marketplaces and spot instances.

Is renting a cloud GPU cheaper than buying?

For occasional use, yes. Buying pays off only if the GPU is busy most of the time: for an RTX 5090 at late-2026 prices, after roughly 4,000–9,000 hours of use compared to renting, for an H100 after more than 10,000 hours, not counting the cost of a server, cooling and maintenance.

Why are AWS, Google Cloud and Azure more expensive?

Hyperscaler GPU instances come with large amounts of CPU and RAM, fast networking and the whole ecosystem of cloud services, security and support, and they sell mostly whole 8-GPU machines. If you only need GPUs, AI-focused clouds are usually two to three times cheaper. Hyperscalers also offer substantial discounts for commitments and credits for startups.

What is a spot or preemptible GPU instance?

It is unused capacity sold at a large discount, often 50–80%, which the provider can take back when someone pays the full price. It works well for jobs that save checkpoints and can resume, and badly for anything that must run without interruption.

Which cloud GPU is best for deep learning?

It depends on model size. For learning and small models, an RTX 4090/5090, L4 or A100 is enough and costs under $2 per hour. For fine-tuning LLMs with up to 32 billion parameters, an A100 80GB, L40S or RTX PRO 6000. For large models and production LLM serving, H100, H200 or B200. Compare the cost per job, not per hour: a faster GPU can finish the job cheaper.