Accuracy is not compatibility: two Evo 2 checkpoints hinge on one FP8 matrix multiplication

Classés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.

Les articles les plus proches

1. Proche Contexte plus large Pas pertinent

   Demystifying Numerical Instability in LLM Inference: Achieving Reproducible Inference for Mission-Critical Tasks with HEAL

   Zhenting Zhu, Lucas Thai, Shan Yu et al. · 2026 · arXiv.org · preprint, non relu par les pairs · cité 3 fois · Pas encore résumé

   HEAL étudie la divergence des sorties entre GPU et attribue l’incohérence aux erreurs de troncature lors des conversions aux frontières des noyaux.

   Résumé scientifique

   As Large Language Models (LLMs) deploy into mission-critical domains (e.g., finance, medicine, and law), output reproducibility has become a strict system requirement. While practitioners use greedy decoding to eliminate algorithmic stochasticity, empirical deployments with 16-bit precisions still exhibit catastrophic output divergence across heterogeneous GPUs. Through SASS-level profiling, we reveal that this inconsistency is fundamentally driven by truncation errors introduced during downcasting at kernel boundaries. However, achieving reproducibility via a global FP32 pipeline incurs prohibitive system penalties: bypassing 16-bit hardware accelerators hurts compute efficiency, while upca…

2. Apparenté Base de cet article Cité par cet article Pas pertinent

   Bit-Accurate Modeling of GPU Matrix Multiply-Accumulate Units: Demystifying Numerical Discrepancy and Accuracy

   Peichen Xie, Shuotao Xu, Yang Wang et al. · 2025 · cité 4 fois · Pas encore résumé

   Ce travail modélise précisément l’arithmétique des unités MMA, sujet que l’étude reprend avec son émulateur pour localiser la dépendance d’Evo 2.

   Résumé scientifique

   Modern AI accelerators rely on matrix multiply-accumulate units (MMAUs), such as NVIDIA Tensor Cores and AMD Matrix Cores, to accelerate deep neural network workloads. MMAUs expose only instruction-level or API-level interfaces of matrix multiply-accumulate (MMA) operations, while leaving internal floating-point arithmetic behavior undocumented. Consequently, MMAUs across vendors and architectural generations often produce numerical discrepancies for identical inputs, and sometimes exhibit reduced numerical accuracy that can cause training instability. Diagnosing and understanding the root causes of these effects is challenging without white-box models of their arithmetic behavior. This pape…

3. Apparenté Contexte plus large Pas pertinent

   CurveFP: Co-Designing Numerical Representation and Product Arithmetic for Language Models

   Ye Qiao · 2026 · Pas encore résumé

   CurveFP conçoit de nouveaux formats et produits arithmétiques pour les modèles de langage, tandis que l’étude mesure la compatibilité de checkpoints FP8 existants.

   Résumé scientifique

   Low-precision formats usually optimize scalar fidelity while inheriting conventional product arithmetic. We introduce CurveFP, a block-scaled family that distributes magnitudes across interleaved logarithmic curves. Uniform curve indices make every nonzero product an exact sign and integer-index update, while a rational radix exposes the finite phase schedule required for accumulation. We instantiate the algebra as CurveFP8 E4C3/E5C2 for training and CurveFP7 E3C3 for compact inference. On four 7B-9B models, CurveFP7 beats tensorwise FP8 perplexity with one fewer element bit and stays within 1.32% of native quality. CurveFP8 lowers error in all 36 paired training-GEMM comparisons. Across thr…

4. Apparenté Contexte plus large Pas pertinent

   Spec Sheets Are Not Kernels: An ISA- and Source-Level Audit of INT8 Availability on NVIDIA Blackwell Ultra

   Teng-Ruei Chen · 2026 · cité 2 fois · Pas encore résumé

   Cet audit examine la disponibilité d’INT8 sur Blackwell, alors que l’étude analyse les effets de l’arithmétique FP8 sur les checkpoints Evo 2.

   Résumé scientifique

   NVIDIA's published specifications give the Blackwell Ultra GPU (B300) a dense-compute ratio of roughly 30:1 between FP8 and INT8 tensor-core throughput; its predecessors, H200 and B200, both provide 1:1. We audit what this deprioritization means in practice by tracing INT8 W8A8 support through four layers of the stack: the published specifications, the PTX ISA, NVIDIA's CUTLASS kernel library, and the two major open-source LLM serving engines (vLLM and SGLang). We find a consistent, layered withdrawal: (i) the PTX ISA never exposes the fifth-generation tensor-core integer path (tcgen05.mma with .kind::i8) on sm_103a, even though the same PTX revision extends the FP4 kinds to that target, lea…

5. Apparenté Données, outil ou méthode en commun Pas pertinent

   Hawkeye: Reproducing GPU-Level Non-Determinism

   Erez Badash, Dan Boneh, Ilan Komargodski et al. · 2026 · arXiv.org · preprint, non relu par les pairs · cité 3 fois · Pas encore résumé

   Hawkeye reproduit sur CPU les opérations arithmétiques GPU, comme mmaemu reproduit les projections FP8 capturées pour examiner leur comportement.

   Résumé scientifique

   We present Hawkeye, a system for analyzing and reproducing GPU-level arithmetic operations. Using our framework, anyone can re-execute on a CPU the exact matrix multiplication operations underlying a machine learning model training or inference workflow that was executed on an NVIDIA GPU, without any precision loss. This is in stark contrast to prior approaches to verifiable machine learning, which either introduce significant computation overhead to the original model owner, or suffer from non-robustness and quality degradation. The main technical contribution of Hawkeye is a systematic sequence of carefully crafted tests that study rounding direction, subnormal number handling, and order o…

6. Apparenté Contexte plus large Pas pertinent

   Reference Traces for Auditing Invisible Weight Updates and Guiding Exact-Budget Protection

   Zekai Shang · 2026 · arXiv.org · preprint, non relu par les pairs · Pas encore résumé

   Cette étude audite les pertes d’updates lors de l’écriture en faible précision pendant l’entraînement, plutôt que les sommes FP8 à l’inférence d’Evo 2.

   Résumé scientifique

   Direct low-precision write-back can erase nonzero optimizer proposals. We ask what a high-precision reference trace establishes before a low-precision run. The exact target-code event is auditable coordinatewise on a realized target trajectory; pre-run aggregate projection also assumes the reference remains a useful counterfactual. In a controlled two-layer grid, 55/72 cells have measured and predicted post-initialization crossings: times span $384\times$, 52/55 are within 15\%, and 4/72 differ in category. Matched decoder experiments show stochastic rather than nearest write-back recovers most of the loss gap. A prospective analytic-grid E4M3 audit reuses one fp32 trace across three unseen …

7. Apparenté Contexte plus large Pas pertinent

   Bridging Consumer and Enterprise: Benchmarking LLaMA, Qwen, and GPT Models on Diverse Hardware

   F. Khan, Shoaib Mustafa Khan · 2026 · International Conference on Multimodal Interaction · Pas encore résumé

   Ce benchmark compare débit et efficacité de modèles de langage sur plusieurs matériels, tandis que l’étude teste la compatibilité numérique d’Evo 2 sur GPU.

   Résumé scientifique

   The increase of large language models (LLMs) such as Qwen, LLaMA, Mistral, and GPT variants has intensified the need for systematic benchmarking across heterogeneous hardware platforms. This paper evaluates NVIDIA A100 and RTX 3090 GPUs, AMD RX 580 and RX 7600 XT GPUs, and an Intel i5-14400F CPU on a diverse suite of open-source LLMs, including Mistral-7B, LLaMA-2 (7B, 13B), LLaMA-3.x (3B–11B), CodeLLaMA, Qwen1.5/2/2.5 (1.8B–14B), and legacy GPT models (GPT-2, GPT-Neo, GPT-J, GPT-NeoX-20B). Performance was assessed in terms of sustained TFLOPs, throughput in tokens per second (TPS), and memory/power efficiency. Results show that smaller Qwen models peaked at 276 TPS (A100), the RTX 3090 domi…

8. Apparenté Autre approche du problème Pas pertinent

   DeepGEMM: Accelerated Ultra Low-Precision Inference on CPU Architectures using Lookup Tables

   Darshan C. Ganji, Saad Ashfaq, Ehsan Saboori et al. · 2023 · Machine Learning · cité 12 fois · Résumé IA disponible

   DeepGEMM accélère les calculs GEMM de modèles quantifiés sur CPU par tables de correspondance, tandis que l’étude restaure les performances d’Evo 2 en changeant l’arithmétique FP8.

   Résumé scientifique

   A lot of recent progress has been made in ultra low-bit quantization, promising significant improvements in latency, memory footprint and energy consumption on edge devices. Quantization methods such as Learned Step Size Quantization can achieve model accuracy that is comparable to full-precision floating-point baselines even with sub-byte quantization. However, it is extremely challenging to deploy these ultra low-bit quantized models on mainstream CPU devices because commodity SIMD (Single Instruction, Multiple Data) hardware typically supports no less than 8-bit precision. To overcome this limitation, we propose DeepGEMM, a lookup table based approach for the execution of ultra low-precis…

Les travaux antérieurs sur lesquels il s’appuie

1. Apparenté Base de cet article Pas pertinent

   Mixed Precision Block Fused Multiply-Add: Error Analysis and Application to GPU Tensor Cores

   Pierre Blanchard, N. Higham, Florent Lopez et al. · 2020 · SIAM Journal on Scientific Computing · cité 71 fois · Pas encore résumé

   Ce travail analyse les erreurs d’unités FMA matricielles mixtes sur Tensor Cores, une base numérique pour étudier l’accumulation FP8.

   Résumé scientifique

   Computing units that carry out a fused multiply-add (FMA) operation with matrix arguments, referred to as tensor units by some vendors, have great potential for use in scientific computing. However...

2. Apparenté Base de cet article Pas pertinent

   Accurate Models of NVIDIA Tensor Cores

   Faizan A. Khattak, M. Mikaitis · 2025 · ACM Transactions on Architecture and Code Optimization (TACO) · cité 8 fois · Pas encore résumé

   Ce travail modélise le comportement numérique des Tensor Cores et les écarts entre générations de GPU, que l’étude examine pour des projections FP8 d’Evo 2.

   Résumé scientifique

   Matrix multiplication is a fundamental operation in both training of neural networks and inference. To accelerate matrix multiplication, Graphical Processing Units (GPUs) provide it implemented in hardware. Due to the increased throughput over the software-based matrix multiplication, the multipliers are increasingly used outside AI, to accelerate various applications in scientific computing. However, matrix multipliers targeted at AI are at present not compliant with IEEE 754 floating-point arithmetic behaviour, with different vendors offering different numerical features. This leads to non-reproducible results across different generations of GPU architectures, at the matrix multiply-accumu…

3. Apparenté Base de cet article Pas pertinent

   Numerical behavior of NVIDIA tensor cores

   Massimiliano Fasi, N. Higham, M. Mikaitis et al. · 2021 · PeerJ Computer Science · cité 78 fois · Pas encore résumé

   Cette étude caractérise expérimentalement l’arithmétique des Tensor Cores NVIDIA, notamment l’arrondi et l’ordre des opérations, comme l’analyse ciblée de l’accumulation FP8.

   Résumé scientifique

   We explore the floating-point arithmetic implemented in the NVIDIA tensor cores, which are hardware accelerators for mixed-precision matrix multiplication available on the Volta, Turing, and Ampere microarchitectures. Using Volta V100, Turing T4, and Ampere A100 graphics cards, we determine what precision is used for the intermediate results, whether subnormal numbers are supported, what rounding mode is used, in which order the operations underlying the matrix multiplication are performed, and whether partial sums are normalized. These aspects are not documented by NVIDIA, and we gain insight by running carefully designed numerical experiments on these hardware units. Knowing the answers to…

4. Apparenté Base de cet article Pas pertinent

   Accurate classification of BRCA1 variants with saturation genome editing

   Gregory M. Findlay, R. Daza, B. Martin et al. · 2018 · Nature · cité 792 fois · Pas encore résumé

   Cette étude établit des effets fonctionnels de variants BRCA1 par édition génomique, tandis que l’article utilise des variants BRCA1 pour évaluer Evo 2.

   Résumé scientifique

   Variants of uncertain significance fundamentally limit the clinical utility of genetic information. The challenge they pose is epitomized by BRCA1, a tumour suppressor gene in which germline loss-of-function variants predispose women to breast and ovarian cancer. Although BRCA1 has been sequenced in millions of women, the risk associated with most newly observed variants cannot be definitively assigned. Here we use saturation genome editing to assay 96.5% of all possible single-nucleotide variants (SNVs) in 13 exons that encode functionally critical domains of BRCA1. Functional effects for nearly 4,000 SNVs are bimodally distributed and almost perfectly concordant with established assessment…

5. Apparenté Base de cet article Pas pertinent

   The pitfalls of verifying floating-point computations

   David Monniaux · 2007 · TOPL · cité 247 fois · Pas encore résumé

   Ce travail expose les difficultés à définir et vérifier la sémantique des calculs flottants, question pertinente à l’étude des écarts d’accumulation FP8.

   Résumé scientifique

   Current critical systems often use a lot of floating-point computations, and thus the testing or static analysis of programs containing floating-point operators has become a priority. However, correctly defining the semantics of common implementations of floating-point is tricky, because semantics may change according to many factors beyond source-code level, such as choices made by compilers. We here give concrete examples of problems that can appear and solutions for implementing in analysis software.