H200 vs B200 vs GB300: Which NVIDIA GPU Should You Rent for AI?

For the first time, three distinct generations of NVIDIA data center silicon are rentable in the cloud at the same time: the Hopper-based H200, the Blackwell B200, and the Blackwell Ultra tier — sold per-GPU as the HGX B300 and at rack scale as the GB300.

That's three real options with meaningfully different memory capacities, precision support, and availability. This guide covers what separates them and how to match each one to a workload. For current hourly rates, use the live GPU pricing table — prices move too fast to print in a blog post.

Quick spec comparison

Bar charts comparing GPU memory and memory bandwidth across the NVIDIA H200, B200, HGX B300, and GB300

Two caveats when reading this table. First, the GB300 column describes a Grace Blackwell superchip — two B300 GPUs plus a Grace CPU on one module — so its memory and throughput figures aren't directly comparable to a single GPU. Second, tensor throughput figures vary by precision and sparsity assumptions across sources; treat them as relative indicators rather than exact benchmarks.

What each one actually is

H200: the Hopper memory refresh

The H200 is the same Hopper compute die as the H100, paired with more and faster memory: 141 GB of HBM3e at 4.8 TB/s, versus the H100's 80 GB at up to 3.35 TB/s. Compute throughput is essentially unchanged.

That makes the H200 an upgrade specifically for memory-bound work — which, in practice, describes most LLM inference. Larger KV caches, longer contexts, and bigger batch sizes fit on a single GPU. A 70B-parameter model in FP8 runs comfortably on one card with room for context.

Because it's Hopper, the software story is fully mature: every framework, inference server, and quantization toolchain has been production-tested on this architecture for years.

B200: the Blackwell generation

The B200 is a genuine architecture change. It's a dual-die design with 180 GB of HBM3e, 8 TB/s of bandwidth — nearly double the H200 — and a large jump in tensor throughput. The headline addition is native FP4 precision, which roughly doubles inference throughput again for models quantized that far, at some accuracy cost that varies by model and workload.

For training, Blackwell's gains are similarly large at FP8, and fifth-generation NVLink improves multi-GPU scaling within a node. The tradeoff is a 1,000 W TDP, which pushes providers toward denser power and cooling — a cost that shows up in hourly rates.

B300 and GB300: Blackwell Ultra

Blackwell Ultra is the memory-and-inference-focused revision of Blackwell, and it comes in two forms that are easy to confuse:

• HGX B300 is the per-GPU form — 288 GB of HBM3e per GPU (up 60% from B200), 8 TB/s bandwidth, and denser FP4 tensor cores. This is what you get when a provider rents you a "B300" instance, typically in 8-GPU nodes.
• GB300 is the Grace Blackwell Ultra superchip: two B300-class GPUs connected to a Grace CPU over NVLink-C2C, giving 576 GB of GPU memory per module. Providers deploy it as the liquid-cooled GB300 NVL72 rack — 72 GPUs and 36 Grace CPUs acting as one large accelerator with 130 TB/s of NVLink bandwidth.

NVIDIA positions Blackwell Ultra at reasoning-model inference: test-time scaling, long chains of thought, and large-context serving all consume memory and interconnect bandwidth faster than they consume raw FLOPs. The 288 GB per GPU means a 400B-class model in FP4 fits on far fewer GPUs, and the NVL72 rack targets models that don't fit in a single node at all.

As of mid-2026 this is NVIDIA's top-tier rentable hardware. If you see "GB300 NVL72" on a provider's site, you're renting slices of that rack, not a single card.

How much model fits on each GPU?

Memory capacity is the spec that most often decides this choice, so it's worth doing the arithmetic explicitly. Model weights need roughly 2 bytes per parameter at FP16, 1 byte at FP8, and half a byte at FP4. On top of the weights, you need room for the KV cache — which grows with batch size and context length — plus activation memory and framework overhead. A common rule of thumb is to keep 20–40% of GPU memory free for the cache and overhead, more if you're serving long contexts at high concurrency.

Applying that math to each tier:

Three practical wrinkles to that table:

• Context length changes everything. Serving a 70B model at 8K context is a very different memory footprint than serving it at 128K. The KV cache for long-context, high-batch serving can exceed the size of the weights themselves — this is exactly the case where the H200's extra 61 GB over the H100, or the B300's extra 108 GB over the B200, converts directly into throughput.
• Fewer GPUs beats more GPUs at the same total memory. Tensor-parallel inference across 4 GPUs pays interconnect and synchronization costs that a single larger-memory GPU doesn't. If a model fits on one B300 but needs two H200s, the B300 option often wins on latency even before considering price.
• Mixture-of-experts models are memory-hungry but compute-light. MoE architectures — increasingly common among open-weight frontier models — need all expert weights resident in memory even though only a fraction activate per token. That skews the decision further toward the high-memory tiers.

Availability: the deciding factor for many teams

Spec sheets don't capture the biggest practical difference — how easy each GPU is to actually get:

Bar chart showing the number of cloud providers with public on-demand listings: 26 for H200, 18 for B200, and 2 for GB300

• H200 is now a commodity. Our pricing data tracks it at 25+ providers, from hyperscalers (AWS, Azure, Oracle) to GPU-focused clouds (CoreWeave, Nebius, Runpod, Lambda) and marketplaces. Wide supply means real price competition and easy multi-region capacity.
• B200 availability has broadened quickly and is available from 18+ providers, mostly GPU-focused clouds. On-demand single-GPU rentals exist, but the best rates cluster around 8-GPU nodes and short-term reservations.
• B300 / GB300 remain scarce. Only a handful of providers list public on-demand pricing; most capacity moves through reserved contracts before it ever hits a pricing page. Expect that to change over the next few quarters, the same way B200 did.

Scarcity shows up directly in hourly rates. Newer generations carry a premium well beyond their performance advantage until supply catches up — which is why per-workload math matters more than per-hour math.

Matching the GPU to the workload

Fine-tuning and serving models up to ~70B parameters: H200. The 141 GB of memory handles a 70B model with a healthy KV cache on a single GPU. Wide availability means you can shop across two dozen providers, and mature Hopper tooling means no surprises. Unless you specifically need FP4 or Blackwell's training throughput, the H200 is the sensible default in 2026.

Large training runs and high-throughput inference: B200. If your job is compute-bound — pretraining, large-scale fine-tuning, or serving at high batch sizes — Blackwell's per-GPU throughput can cut total GPU-hours enough to offset a higher hourly rate. Doing the math per token or per training step, rather than per hour, is the right comparison, and it increasingly favors B200 as its price gap versus H200 narrows.

Frontier-scale and reasoning-model inference: B300 / GB300. Models in the several-hundred-billion-parameter range, very long contexts, or reasoning workloads that scale test-time compute are what this tier is built for. The 288 GB per GPU shrinks cluster sizes, and the NVL72 rack removes node boundaries entirely. The practical constraints are availability and commitment: much of this capacity requires reservations rather than on-demand hourly rental.

Budget-sensitive experimentation: none of the above. If your models fit in 80 GB, the H100 — and even the A100 — have become substantially cheaper as demand migrated up the stack. The H100 across providers is often the best value for smaller-scale work, exactly because it's no longer the newest thing.

What about the H100 and GB200?

Two adjacent options are worth situating, since they bracket the three tiers above.

The H100 remains the most widely offered data center GPU in the cloud, and the migration of demand toward H200 and Blackwell has steadily pushed its rates down. If your workload fit an H100 in 2024, nothing about it has gotten worse — it's the same silicon at a better price. The main reasons to move up are memory (80 GB vs 141 GB) and, for inference at scale, bandwidth.

The GB200 is the original Grace Blackwell superchip — two B200-class GPUs plus a Grace CPU, deployed as the GB200 NVL72 rack. It sits between B200 and GB300 in both capability and availability. Providers that invested in GB200 NVL72 racks in 2025 still rent them, and for workloads that need rack-scale NVLink but not Blackwell Ultra's extra memory, they can be the more available option of the two superchip generations.

Three practical notes before you rent

1. Per-GPU pricing varies by node size. Many providers price 8-GPU nodes differently than single GPUs, and B300/GB300 capacity is often only sold in full nodes or racks. Check the

   gpu_count

   on any listing — our tables always normalize to per-GPU rates.
2. FP4 is an inference feature, not a free lunch. Blackwell's FP4 throughput numbers assume your model quantizes to 4-bit acceptably. For many workloads it does; for some it doesn't. Benchmark your own model before letting an FP4 spec sheet drive a hardware decision.
3. Same CUDA, different maturity. All three generations run the same software stack, but newer silicon takes time to reach peak framework performance. Hopper is fully optimized; Blackwell Ultra kernels are still improving quarter over quarter, which means rented B300 capacity tends to get faster over its lifetime.

Frequently asked questions

Is the H200 worth it over the H100? For inference, usually yes — the 141 GB vs 80 GB memory difference determines whether a 70B model with real context fits on one GPU, and 4.8 TB/s vs 3.35 TB/s of bandwidth translates directly into token throughput for memory-bound serving. For compute-bound training, the two perform similarly, so the decision comes down to the current price gap on the pricing table.

What's the difference between B300, HGX B300, and GB300? They share the same Blackwell Ultra GPU. "B300" or "HGX B300" means the per-GPU form with 288 GB, sold in standard 8-GPU nodes. "GB300" is the superchip pairing two of those GPUs with a Grace CPU, and "GB300 NVL72" is the 72-GPU liquid-cooled rack built from those superchips. Cloud listings use all three names — check the memory figure to confirm what you're getting.

Can a B200 replace two H200s? For compute-bound work, one B200 delivers several times an H200's tensor throughput, so often yes. For memory-bound work, no: a B200's 180 GB is less than two H200s' 282 GB. If your bottleneck is fitting the model plus a large KV cache, count gigabytes rather than TFLOPS.

Do I need NVLink, or are PCIe/Ethernet clusters fine? Single-GPU inference doesn't care. Multi-GPU tensor parallelism cares a lot — activations cross the interconnect every layer, and NVLink's bandwidth advantage over PCIe shows up directly in latency. Training across nodes depends more on the cluster fabric (InfiniBand vs Ethernet) than on the GPU generation, which is a provider-level question worth asking before committing.

When does renting Blackwell Ultra start making financial sense? When the alternative is stringing together enough H200s or B200s that interconnect overhead, extra nodes, and engineering time cost more than the newer hardware's premium — or when your model simply doesn't fit anything smaller. Scarcity pricing fades as supply ramps; the B200's trajectory over 2025 is the template. Watch the live listings rather than assuming today's premium is permanent.

Which one is cheapest per hour? That changes weekly and varies several-fold across providers for the same GPU, which is why we don't print prices in posts. The GPU comparison table shows current per-GPU rates for every provider we track, filterable by GPU model.

Bottom line

The H200 is the widely available, well-understood default for most inference and fine-tuning. The B200 wins when raw throughput per GPU determines your total bill. Blackwell Ultra — B300 per-GPU, GB300 at rack scale — exists for workloads where memory capacity and interconnect are the binding constraint, and its scarcity premium will keep narrowing as supply ramps.

Whatever tier fits, compare live rates across providers before committing: the spread between the cheapest and most expensive listing for the same GPU is routinely several-fold. Current prices for every provider we track are on the GPU comparison page, with per-GPU detail on the H200, B200, HGX B300, and GB300 pages.