Matching Workloads to Blackwell: Power-Resolved Benchmarking of NVIDIA B200 and B300 across HPC and AI Workloads

Classés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.

Les articles les plus proches

1. Proche Contexte plus large Pas pertinent

   In-Datacenter Performance Analysis of a Tensor Processing Unit

   Norman P. Jouppi, Cliff Young, Nishant Patil et al. · 2017 · Hardware Architecture · cité 5 584 fois · Résumé IA disponible

   In-Datacenter Performance Analysis évalue un ASIC TPU pour l’inférence de réseaux neuronaux, tandis que l’article compare des GPU sur des charges d’IA et de calcul scientifique.

   Résumé scientifique

   Many architects believe that major improvements in cost-energy-performance must now come from domain-specific hardware. This paper evaluates a custom ASIC---called a Tensor Processing Unit (TPU)---deployed in datacenters since 2015 that accelerates the inference phase of neural networks (NN). The heart of the TPU is a 65,536 8-bit MAC matrix multiply unit that offers a peak throughput of 92 TeraOps/second (TOPS) and a large (28 MiB) software-managed on-chip memory. The TPU's deterministic execution model is a better match to the 99th-percentile response-time requirement of our NN applications than are the time-varying optimizations of CPUs and GPUs (caches, out-of-order execution, multithrea…

2. Proche Contexte plus large Pas pertinent

   Benchmarking CPU, GPU, and ML Power-Flow Solvers from 14 to 70,000 Buses

   Benjamin Donnot, Sami Tazi, Eva Boguslawski et al. · 2026 · Distributed, Parallel, and Cluster Computing · Résumé IA disponible

   Benchmarking CPU, GPU, and ML Power-Flow Solvers compare des solveurs pour le calcul de flux de puissance, alors que l’article évalue des GPU sur plusieurs charges HPC et d’IA.

   En bref

   Sur les cas étudiés, des solveurs physiques CPU parallélisés ou GPU égalent ou dépassent le débit des substituts neuronaux évalués, avec une précision nettement supérieure.

3. Proche Même question Pas pertinent

   From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inference

   Siddharth Samsi, Dan Zhao, Joseph McDonald et al. · 2023 · Computation and Language · cité 376 fois · Résumé IA disponible

   From Words to Watts mesure les coûts énergétiques de l’inférence LLM, tandis que l’article mesure notamment la puissance et le débit de Llama-3 sur B200 et B300.

   Résumé scientifique

   Large language models (LLMs) have exploded in popularity due to their new generative capabilities that go far beyond prior state-of-the-art. These technologies are increasingly being leveraged in various domains such as law, finance, and medicine. However, these models carry significant computational challenges, especially the compute and energy costs required for inference. Inference energy costs already receive less attention than the energy costs of training LLMs -- despite how often these large models are called on to conduct inference in reality (e.g., ChatGPT). As these state-of-the-art LLMs see increasing usage and deployment in various domains, a better understanding of their resourc…

4. Proche Même question Pas pertinent

   Not All GPUs Are Created Equal: Characterizing Variability in Large-Scale, Accelerator-Rich Systems

   Prasoon Sinha, Akhil Guliani, Rutwik Jain et al. · 2022 · Distributed, Parallel, and Cluster Computing · cité 47 fois · Pas encore résumé

   Not All GPUs Are Created Equal étudie les variations de performance entre accélérateurs de même architecture, tandis que l’article observe des écarts entre clusters B300 identiques.

   Résumé scientifique

   Scientists are increasingly exploring and utilizing the massive parallelism of general-purpose accelerators such as GPUs for scientific breakthroughs. As a result, datacenters, hyperscalers, national computing centers, and supercomputers have procured hardware to support this evolving application paradigm. These systems contain hundreds to tens of thousands of accelerators, enabling peta- and exa-scale levels of compute for scientific workloads. Recent work demonstrated that power management (PM) can impact application performance in CPU-based HPC systems, even when machines have the same architecture and SKU (stock keeping unit). This variation occurs due to manufacturing variability and th…

5. Proche Même méthode, autre problème Pas pertinent

   MIGPerf: A Comprehensive Benchmark for Deep Learning Training and Inference Workloads on Multi-Instance GPUs

   Huaizheng Zhang, Yuanming Li, Wencong Xiao et al. · 2023 · Machine Learning · preprint, non relu par les pairs · cité 10 fois · Pas encore résumé

   MIGPerf benchmarke entraînement et inférence de vision sur GPU, comme l’article, mais étudie le partitionnement MIG plutôt que les B200 et B300.

   Résumé scientifique

   New architecture GPUs like A100 are now equipped with multi-instance GPU (MIG) technology, which allows the GPU to be partitioned into multiple small, isolated instances. This technology provides more flexibility for users to support both deep learning training and inference workloads, but efficiently utilizing it can still be challenging. The vision of this paper is to provide a more comprehensive and practical benchmark study for MIG in order to eliminate the need for tedious manual benchmarking and tuning efforts. To achieve this vision, the paper presents MIGPerf, an open-source tool that streamlines the benchmark study for MIG. Using MIGPerf, the authors conduct a series of experiments,…

6. Proche Même méthode, autre problème Pas pertinent

   Benchmarking and Dissecting the Nvidia Hopper GPU Architecture

   Weile Luo, Ruibo Fan, Zeyu Li et al. · 2024 · Hardware Architecture · cité 110 fois · Résumé IA disponible

   Benchmarking and Dissecting caractérise l’architecture Hopper par des benchmarks GPU, tandis que l’article compare les performances de GPU Blackwell sur diverses charges.

   Résumé scientifique

   Graphics processing units (GPUs) are continually evolving to cater to the computational demands of contemporary general-purpose workloads, particularly those driven by artificial intelligence (AI) utilizing deep learning techniques. A substantial body of studies have been dedicated to dissecting the microarchitectural metrics characterizing diverse GPU generations, which helps researchers understand the hardware details and leverage them to optimize the GPU programs. However, the latest Hopper GPUs present a set of novel attributes, including new tensor cores supporting FP8, DPX, and distributed shared memory. Their details still remain mysterious in terms of performance and operational char…

7. Apparenté Contexte plus large Pas pertinent

   Efficient Large-Scale Language Model Training on GPU Clusters

   Deepak Narayanan, Mohammad Shoeybi, Jared Casper et al. · 2021 · Computation and Language · cité 1 465 fois · Résumé IA disponible

   Efficient Large-Scale Language Model Training traite du parallélisme et de la mise à l’échelle de l’entraînement LLM sur clusters GPU, une charge proche de celles évaluées dans l’article.

   Résumé scientifique

   Large language models have led to state-of-the-art accuracies across a range of tasks. However, training these large models efficiently is challenging for two reasons: a) GPU memory capacity is limited, making it impossible to fit large models on a single GPU or even on a multi-GPU server; and b) the number of compute operations required to train these models can result in unrealistically long training times. New methods of model parallelism such as tensor and pipeline parallelism have been proposed to address these challenges; unfortunately, naive usage leads to fundamental scaling issues at thousands of GPUs due to various reasons, e.g., expensive cross-node communication or idle periods w…

8. Apparenté Même question Pas pertinent

   DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiency

   Jovan Stojkovic, Chaojie Zhang, Íñigo Goiri et al. · 2024 · Artificial Intelligence · cité 226 fois · Résumé IA disponible

   DynamoLLM étudie la performance et l’efficacité énergétique de l’inférence LLM sur GPU, tandis que l’article mesure puissance et débit pour Llama-3 sur B200 et B300.

   Résumé scientifique

   The rapid evolution and widespread adoption of generative large language models (LLMs) have made them a pivotal workload in various applications. Today, LLM inference clusters receive a large number of queries with strict Service Level Objectives (SLOs). To achieve the desired performance, these models execute on power-hungry GPUs causing the inference clusters to consume large amount of energy and, consequently, result in excessive carbon emissions. Fortunately, we find that there is a great opportunity to exploit the heterogeneity in inference compute properties and fluctuations in inference workloads, to significantly improve energy-efficiency. However, such a diverse and dynamic environm…