Matching Workloads to Blackwell: Power-Resolved Benchmarking of NVIDIA B200 and B300 across HPC and AI Workloads

Mohsen Zand; Trung Dac Nguyen; H. Birali Runesha

AI summary · arXiv: 2610.41919v1 · cs.DC

Matching Workloads to Blackwell: Power-Resolved Benchmarking of NVIDIA B200 and B300 across HPC and AI Workloads

Mohsen Zand, Trung Dac Nguyen, H. Birali Runesha

Original paper Submitted to arXiv on 30 Oct. 2026 10 pages, 8 figures. Accepted at the 17th International Workshop on Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems (PMBS 2026), held in conjunction with SC26 License: NONEXCLUSIVE-DISTRIB 1.0 Also in: fr

How to read this summary? Illustrated guide

The essentials

Across AI and HPC benchmarks, B300 does not uniformly outperform B200: advantages depend on workload, while kernel granularity, double-precision capability and software maturity shape substantial regressions.

Question

Does NVIDIA B300 improve application performance and energy efficiency over B200 across research workloads? The study examines whether observed differences reflect hardware capabilities, workload execution patterns or software-stack maturity.

Method

The authors benchmarked B200 and B300 on two independently administered eight-GPU clusters, covering computer vision, Llama-3-8B and 70B inference, Pangu-Weather training and six production HPC codes. Computer-vision inputs remained in device memory; inference used 100 timed iterations after 30 warmups, training used 50 after 15, and rank-0 board power was sampled at 20 Hz. LLM tests used 150 prompts with generation capped at 256 tokens; CV and LLM configurations had single timed runs, whereas HPC and 3D transformer results were medians of five runs.

Findings

Result At eight GPUs in BF16, B200 delivered 5.6 times the ConvNeXt-Base inference throughput of B300, whereas B300 led ViT-Large inference by 21%.
See the passage in the paper
At 8 GPUs in BF16, ResNet-50 reaches 110.7 K FPS on B200 against 35.6 K on B300 (3.1$\times$), and ConvNeXt-Base 52.9 K against 9.4 K (5.6$\times$). EfficientNet-B4 is nearly level (45.8 K vs. 39.0 K).
Authors’ interpretation The authors interpret low B300 power draw on poorly performing convolutional workloads as evidence of per-kernel costs rather than input-data supply, without resolving launch overhead versus immature convolution kernels.
See the passage in the paper
Two candidates remain: CPU-side kernel-launch overhead, which dominates for launch-heavy networks of many small convolutions, and the maturity of sm_103 convolution kernels in cuDNN [11] 9.15.
Result B200 led 70B LLM throughput by 24–27%, while the authors attributed substantially higher time-to-first-token on one B300 system to its development vLLM build.
See the passage in the paper
For Llama-3-8B at 8 GPUs, the B200 leads by 16% on Lenovo (24.2 vs. 20.9 K tok/s); for Llama-3-70B, it leads by 24–27% (11.0–11.2 K vs. 8.7–9.0 K). Tensor parallelism scales poorly at 8B, yielding only 12–30% additional throughput for 8$\times$ the hardware, since TP targets latency rather than aggregate throughput; throughput-oriented deployments at this scale should prefer data-parallel replicas.
Result Pangu-Weather training reached 8.3 samples per second on B200 versus 1.0 on B300 at global batch size 12 over eight GPUs.
See the passage in the paper
Performance peaks at 4 GPUs and regresses at 8, and FP8 on the B300 is slower than BF16, which we attribute to the same immature-stack effect.
Result B300's double-precision LAMMPS performance was 2.6–2.7 times lower than H200 across all five tested system sizes.
See the passage in the paper
For LAMMPS [15] with KOKKOS [16] on the Lennard-Jones benchmark, B300 double-precision performance is 2.6–2.7$\times$ lower than H200 across all five system sizes; B300 single-precision merely matches H200 double-precision.

Limitations

CV and LLM results use single timed runs with spot-checked variation of 2–4%, limiting interpretation of small differences.
See the passage in the paper
First, replication is uneven, as set out in Section III: the HPC and 3D ViT figures are medians of five runs, while the CV and LLM figures are single runs with spot-checked variation of 2–4%.
The proposed convolutional diagnosis does not distinguish CPU-side launch overhead from immature convolution kernels, because the discriminating experiments were not performed.
See the passage in the paper
We are explicit that our data do not separate the two, and we name the discriminating experiment rather than defer it vaguely.
The device-resident CV harness excludes input-pipeline bottlenecks that may affect real deployments.
See the passage in the paper
Second, our CV harness deliberately excludes the input pipeline. A deployment feeding real images from storage may well encounter a genuine data-supply bottleneck that our measurement is blind to by construction.

Why it matters

The authors argue that Blackwell Ultra procurement should be workload-specific rather than treating B300 as a drop-in successor to B200. They recommend profiling kernel granularity, measuring power alongside throughput and benchmarking the exact deployment software build, preferably across independently administered systems.

Written by AI from the paper. Passages are checked word for word against the text we analysed; the rest may contain errors.

AI-generated keywords: NVIDIA Blackwell GPU benchmarking Scientific computing AI workloads Software maturity

Read further

Comprehensive Summary
Layman's Summary
Blog article
Original abstract

This paper examines whether NVIDIA’s B300 Blackwell Ultra GPU consistently outperforms the B200 in scientific computing and AI, and whether observed differences stem from hardware capabilities or software maturity. Its central finding is that the B300 is not a uniform upgrade: greater memory capacity and advantages on some transformer workloads coexist with substantial performance regressions on convolutional networks, a weather-model transformer, and double-precision computations. The authors benchmarked eight-GPU B200 and B300 systems across two independently administered clusters, using older NVIDIA GPUs as additional baselines. Workloads included four computer-vision models, distributed training, Llama-3-8B and 70B inference, Pangu-Weather training, and several scientific applications. The Blackwell systems used CUDA 13.2 and matched core libraries, although drivers, host allocations, and vLLM builds differed. Computer-vision inputs remained in GPU memory, excluding storage access, decoding, and host-to-device transfers from timed measurements. Alongside throughput, the authors sampled one GPU’s board power at 20 Hz. Computer-vision and LLM configurations were measured in single timed runs, with spot checks indicating 2–4% variability. Scientific application and Pangu-Weather results were reported as medians of five runs. Computer-vision performance varied substantially by workload. With eight GPUs in BF16, the B200 achieved 110,700 images per second on ResNet-50, compared with 35,600 for the B300—approximately 3.1 times the throughput. On ConvNeXt-Base, the B200 reached 52,900 images per second versus 9,400 for the B300, a 5.6-fold advantage. EfficientNet-B4 showed a smaller gap, at 45,800 versus 39,000 images per second. In contrast, the B300 improved ViT-Large inference throughput by 21%, reaching 47,700 versus 39,500 images per second, and increased training throughput by 16%. Power measurements reinforced this contrast. The B300 used only 36–53% of its 1,100-watt power envelope on the two convolutional workloads with the largest performance deficits, but approximately 95% on ViT-Large inference. These observations suggest GPU underutilization rather than an input-supply bottleneck, although they do not distinguish kernel-launch overhead from inefficient library kernels. For Llama-3 inference, the B200 generally led, delivering approximately 24–27% higher throughput on the 70B model in the reported comparisons. However, precision and software configuration substantially affected performance. On the B300 system using the production software stack, FP8 inference throughput for the 8B model exceeded the corresponding B200 result by 19%. The B300 system using the development stack had a time to first token up to 2.1 times as long as that of the other B300 system, highlighting software provenance as an important confounding factor. Pangu-Weather training showed a much larger regression: at a global batch size of 12 across eight GPUs, the B200 delivered 8.3 samples per second, compared with 1.0 for the B300. Scientific workloads further differentiated the devices. Quantum ESPRESSO’s dominant double-precision kernel ran 16 times slower on the B300 than on the B200, consistent with the B300’s much lower specified FP64 capability. Double-precision LAMMPS ran 2.6–2.7 times slower on the B300 than on the H200. By contrast, four-GPU GROMACS performance differed by less than 1% between the B200 and B300, with both delivering approximately 1.9 times the performance of the H200. The authors recommend workload-specific procurement rather than treating the B300 as a drop-in replacement for the B200. The B300 is attractive for memory-capacity-constrained applications and the tested large-matrix transformer workloads, whereas the B200 is preferable for the tested launch-heavy models and FP64 applications. Important limitations include uneven replication, power measurements from only one GPU, incomplete B300 coverage for some scientific codes, and toolchain differences in comparisons with older GPUs. The proposed kernel-granularity explanation remains provisional; CUDA graph experiments or execution timelines are needed to establish the underlying mechanism.

NVIDIA B300 vs. B200: Why Newer Does Not Always Mean Faster

A newer GPU might seem like an obvious upgrade, especially when it offers greater memory capacity. But scientific computing and AI workloads do not all benefit from the same hardware features. A benchmark study of NVIDIA’s B300 Blackwell Ultra and B200 shows just how consequential those differences can be.

In “Matching Workloads to Blackwell: Power-Resolved Benchmarking of NVIDIA B200 and B300 across HPC and AI Workloads,” Mohsen Zand, Trung Dac Nguyen, and H. Birali Runesha find that the B300 is far from a universal replacement for the B200. It improves performance on some transformer workloads, but falls substantially behind on several convolutional models, weather-model training, and double-precision calculations.

What the Researchers Tested

The study benchmarked eight-GPU B200 and B300 systems across two independently administered clusters, with older NVIDIA GPUs providing additional comparisons. Tests covered four computer-vision models, distributed training, Llama-3-8B and 70B inference, Pangu-Weather training, and scientific applications.

The Blackwell systems used CUDA 13.2 and matched core libraries, although drivers, host allocations, and vLLM builds differed. These distinctions matter: the study investigates not only which GPU performs better, but also whether performance gaps reflect hardware capabilities or the maturity of the software running on them.

For computer vision, inputs stayed in GPU memory, keeping storage access, image decoding, and host-to-device transfers outside the timed measurements. The researchers also sampled one GPU’s board power at 20 Hz to help interpret utilization.

Computer Vision Reveals a Split

The largest computer-vision gaps favored the B200. Across eight GPUs using BF16, it processed 110,700 images per second on ResNet-50, compared with 35,600 on the B300—roughly a 3.1-fold advantage. On ConvNeXt-Base, the gap widened to 5.6-fold: 52,900 versus 9,400 images per second.

EfficientNet-B4 showed a smaller difference, with the B200 reaching 45,800 images per second against the B300’s 39,000. Yet the direction reversed for ViT-Large. There, the B300 delivered 47,700 images per second in inference, a 21% improvement over the B200’s 39,500. Its training throughput was also 16% higher.

What Power Measurements Suggest

On the two convolutional workloads with the largest deficits, the B300 used only 36–53% of its 1,100-watt power envelope. During ViT-Large inference, it reached approximately 95%.

Because the inputs were already GPU-resident, these observations point toward underutilization rather than a storage or data-transfer bottleneck. However, they do not establish the cause. Kernel-launch overhead and inefficient library kernels remain possible explanations, not proven diagnoses.

Transformers Do Not Tell a Single Story

The B300’s ViT-Large gains did not translate into consistent advantages across other transformer workloads. In the reported Llama-3-70B comparisons, the B200 generally delivered approximately 24–27% higher inference throughput.

Precision and software configuration changed the picture. Using the production software stack, the B300 exceeded the corresponding B200 result by 19% for Llama-3-8B FP8 inference. Meanwhile, the B300 system running a development stack had a time to first token up to 2.1 times as long as the other B300 system. Software provenance is therefore an important confounding factor.

Pangu-Weather produced a much larger regression. At a global batch size of 12 across eight GPUs, the B200 achieved 8.3 samples per second, while the B300 managed 1.0. The result underscores why “transformer workload” is too broad a category to predict performance reliably.

Scientific Computing Depends on Precision

For double-precision applications, the B300’s lower specified FP64 capability had clear consequences. Quantum ESPRESSO’s dominant double-precision kernel ran 16 times slower on the B300 than on the B200. Double-precision LAMMPS ran 2.6–2.7 times slower on the B300 than on the H200.

Other scientific workloads behaved differently. Four-GPU GROMACS performance differed by less than 1% between the B200 and B300, with both delivering approximately 1.9 times the H200’s performance. Scientific computing, like AI, cannot be reduced to one representative benchmark.

How Much Confidence Should Buyers Place in the Results?

The largest gaps are striking, but the study has limitations. Computer-vision and LLM configurations used single timed runs, with spot checks suggesting 2–4% variability. Scientific application and Pangu-Weather results were medians of five runs. Power sampling covered only one GPU, some scientific codes lacked complete B300 coverage, and comparisons with older GPUs involved toolchain differences.

The proposed kernel-granularity explanation also remains provisional. CUDA graph experiments or execution timelines would help distinguish launch overhead from other sources of inefficiency.

The Practical Takeaway: Match the GPU to the Job

The B300 is attractive when memory capacity is the limiting factor and for the tested large-matrix transformer workloads.
The B200 is preferable for the tested launch-heavy models and FP64 applications.
Procurement decisions should include benchmarks using the intended precision, workload configuration, and production software stack.

The central lesson is not that the B300 is a poor GPU. It is that a newer generation does not guarantee a faster application. Hardware capabilities, workload structure, and software maturity must be evaluated together.

Created on 02 Nov. 2026

Was this synthesis useful?

Score: 0

Ask questions about this paper to our AI assistant

How to chat with this paper? Illustrated guide

Compare with the PDF

You can also chat with multiple papers at once here.

AI assistant instructions?

Tools for this paper

Explain it differently 3 free rewrites left today

Save to a collection Group papers in collections to compare them, chat with them and export them. Create a free account Log in

Export BibTeX · RIS (Zotero, Mendeley, EndNote) · Obsidian (Markdown) · Notion (Markdown)

Advanced tools Premium

For researchers who follow a field closely: citation alerts, journal-club slides, version comparison and Zotero export. Discover Premium

Zotero

Citation alerts Citation alerts guide

Who works on it? The people and institutions who publish on its topic

Weekly letter Presented in "Computing beyond AI this week" of 2 November 2026. Computing beyond AI this week: Better algorithms, smaller machines, harder tests

A rising topic This paper is part of "Making AI cheaper and faster to run", rising in Computer Science (cs) (×7.3 compared with usual). The rising topics of that week · The history of the term

Journal club Premium+ Slides to present it Propose for the journal club

Versions What changed between versions

Talk about it in public Premium+ Press kit: release, questions, hype check Press kit guide

Papers around this one

See them on a map

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

Close Broader context
In-Datacenter Performance Analysis of a Tensor Processing Unit
Norman P. Jouppi, Cliff Young, Nishant Patil et al. · 2017 · Hardware Architecture · cited 5,584 times · AI summary available

The TPU study measures datacenter neural-network inference performance and energy, while this paper benchmarks Blackwell GPUs across AI and HPC workloads.

Abstract
Many architects believe that major improvements in cost-energy-performance must now come from domain-specific hardware. This paper evaluates a custom ASIC---called a Tensor Processing Unit (TPU)---deployed in datacenters since 2015 that accelerates the inference phase of neural networks (NN). The heart of the TPU is a 65,536 8-bit MAC matrix multiply unit that offers a peak throughput of 92 TeraOps/second (TOPS) and a large (28 MiB) software-managed on-chip memory. The TPU's deterministic execution model is a better match to the 99th-percentile response-time requirement of our NN applications than are the time-varying optimizations of CPUs and GPUs (caches, out-of-order execution, multithrea…
Close Broader context
Benchmarking CPU, GPU, and ML Power-Flow Solvers from 14 to 70,000 Buses
Benjamin Donnot, Sami Tazi, Eva Boguslawski et al. · 2026 · Distributed, Parallel, and Cluster Computing · AI summary available

The power-flow study compares CPU, GPU, and learned solvers for grid computation, unlike this paper’s Blackwell hardware and workload benchmarking.

In short
Optimized parallel CPU and batched GPU physics solvers match or exceed tested GNN power-flow surrogates in throughput while delivering substantially higher accuracy.
Close Same question
From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inference
Siddharth Samsi, Dan Zhao, Joseph McDonald et al. · 2023 · Computation and Language · cited 376 times · AI summary available

From Words to Watts measures the energy costs of LLM inference, while this paper evaluates Llama inference throughput and GPU power.

Abstract
Large language models (LLMs) have exploded in popularity due to their new generative capabilities that go far beyond prior state-of-the-art. These technologies are increasingly being leveraged in various domains such as law, finance, and medicine. However, these models carry significant computational challenges, especially the compute and energy costs required for inference. Inference energy costs already receive less attention than the energy costs of training LLMs -- despite how often these large models are called on to conduct inference in reality (e.g., ChatGPT). As these state-of-the-art LLMs see increasing usage and deployment in various domains, a better understanding of their resourc…
Close Same question
Not All GPUs Are Created Equal: Characterizing Variability in Large-Scale, Accelerator-Rich Systems
Prasoon Sinha, Akhil Guliani, Rutwik Jain et al. · 2022 · Distributed, Parallel, and Cluster Computing · cited 47 times · Not summarized yet

Not All GPUs Are Created Equal studies performance variation among accelerator-rich systems, while this paper compares performance and power across B200 and B300 clusters.

Abstract
Scientists are increasingly exploring and utilizing the massive parallelism of general-purpose accelerators such as GPUs for scientific breakthroughs. As a result, datacenters, hyperscalers, national computing centers, and supercomputers have procured hardware to support this evolving application paradigm. These systems contain hundreds to tens of thousands of accelerators, enabling peta- and exa-scale levels of compute for scientific workloads. Recent work demonstrated that power management (PM) can impact application performance in CPU-based HPC systems, even when machines have the same architecture and SKU (stock keeping unit). This variation occurs due to manufacturing variability and th…
Close Shared data, tool or method
MIGPerf: A Comprehensive Benchmark for Deep Learning Training and Inference Workloads on Multi-Instance GPUs
Huaizheng Zhang, Yuanming Li, Wencong Xiao et al. · 2023 · Machine Learning · preprint, not peer reviewed · cited 10 times · Not summarized yet

MIGPerf provides a benchmark tool and experiments for GPU training and inference, whereas this paper benchmarks workloads on B200 and B300.

Abstract
New architecture GPUs like A100 are now equipped with multi-instance GPU (MIG) technology, which allows the GPU to be partitioned into multiple small, isolated instances. This technology provides more flexibility for users to support both deep learning training and inference workloads, but efficiently utilizing it can still be challenging. The vision of this paper is to provide a more comprehensive and practical benchmark study for MIG in order to eliminate the need for tedious manual benchmarking and tuning efforts. To achieve this vision, the paper presents MIGPerf, an open-source tool that streamlines the benchmark study for MIG. Using MIGPerf, the authors conduct a series of experiments,…
Close Same method, another problem
Benchmarking and Dissecting the Nvidia Hopper GPU Architecture
Weile Luo, Ruibo Fan, Zeyu Li et al. · 2024 · Hardware Architecture · cited 110 times · AI summary available

Benchmarking and Dissecting the Nvidia Hopper GPU Architecture characterizes GPU performance and architecture, a hardware-benchmarking approach also used here for Blackwell.

Abstract
Graphics processing units (GPUs) are continually evolving to cater to the computational demands of contemporary general-purpose workloads, particularly those driven by artificial intelligence (AI) utilizing deep learning techniques. A substantial body of studies have been dedicated to dissecting the microarchitectural metrics characterizing diverse GPU generations, which helps researchers understand the hardware details and leverage them to optimize the GPU programs. However, the latest Hopper GPUs present a set of novel attributes, including new tensor cores supporting FP8, DPX, and distributed shared memory. Their details still remain mysterious in terms of performance and operational char…
Related Broader context
Efficient Large-Scale Language Model Training on GPU Clusters
Deepak Narayanan, Mohammad Shoeybi, Jared Casper et al. · 2021 · Computation and Language · cited 1,465 times · AI summary available

Efficient Large-Scale Language Model Training on GPU Clusters addresses distributed LLM training efficiency, while this paper benchmarks Pangu-Weather training and other workloads on GPUs.

Abstract
Large language models have led to state-of-the-art accuracies across a range of tasks. However, training these large models efficiently is challenging for two reasons: a) GPU memory capacity is limited, making it impossible to fit large models on a single GPU or even on a multi-GPU server; and b) the number of compute operations required to train these models can result in unrealistically long training times. New methods of model parallelism such as tensor and pipeline parallelism have been proposed to address these challenges; unfortunately, naive usage leads to fundamental scaling issues at thousands of GPUs due to various reasons, e.g., expensive cross-node communication or idle periods w…
Related Same question
DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiency
Jovan Stojkovic, Chaojie Zhang, Íñigo Goiri et al. · 2024 · Artificial Intelligence · cited 226 times · AI summary available

DynamoLLM studies performance and energy efficiency in LLM inference clusters, while this paper measures LLM inference throughput and GPU power.

Abstract
The rapid evolution and widespread adoption of generative large language models (LLMs) have made them a pivotal workload in various applications. Today, LLM inference clusters receive a large number of queries with strict Service Level Objectives (SLOs). To achieve the desired performance, these models execute on power-hungry GPUs causing the inference clusters to consume large amount of energy and, consequently, result in excessive carbon emissions. Fortunately, we find that there is a great opportunity to exploit the heterogeneity in inference compute properties and fluctuations in inference workloads, to significantly improve energy-efficiency. However, such a diverse and dynamic environm…

Go further

How are these papers found?

We compare the title and abstract of this paper with those of every paper summarized on this site, and with the papers Semantic Scholar recommends, those it cites and those citing it (for a much-cited paper, its most recent citations and, from OpenAlex, the most cited ones). Among the prior work and the follow-ups, the papers that use each other in depth, a method it uses and the much-cited follow-ups come a little higher. The levels (very close, close, related) come from the similarity of the texts, not from a judgment of quality; * marks a paper without an abstract, compared by its title. The sentence under a paper is written by an AI from the two abstracts: check it in the papers. A paper marked not relevant by three readers leaves the list.

Updated on 2 November 2026.

Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.

Matching Workloads to Blackwell: Power-Resolved Benchmarking of NVIDIA B200 and B300 across HPC and AI Workloads

The essentials

Question

Method

Findings

Limitations

Why it matters

Read further

Definitions

NVIDIA B300 vs. B200: Why Newer Does Not Always Mean Faster

What the Researchers Tested

Computer Vision Reveals a Split

What Power Measurements Suggest

Transformers Do Not Tell a Single Story

Scientific Computing Depends on Precision

How Much Confidence Should Buyers Place in the Results?

The Practical Takeaway: Match the GPU to the Job

Ask questions about this paper to our AI assistant

Tools for this paper

Advanced tools Premium

Papers around this one

Closest papers

Go further

Cite this paper (BibTeX)