Accuracy is not compatibility: two Evo 2 checkpoints hinge on one FP8 matrix multiplication

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

1. Close Broader context Not relevant

   Demystifying Numerical Instability in LLM Inference: Achieving Reproducible Inference for Mission-Critical Tasks with HEAL

   Zhenting Zhu, Lucas Thai, Shan Yu et al. · 2026 · arXiv.org · preprint, not peer reviewed · cited 3 times · Not summarized yet

   HEAL traces divergent LLM outputs across GPUs to truncation at kernel boundaries, while this paper localizes checkpoint failures to FP8 accumulation arithmetic.

   Abstract

   As Large Language Models (LLMs) deploy into mission-critical domains (e.g., finance, medicine, and law), output reproducibility has become a strict system requirement. While practitioners use greedy decoding to eliminate algorithmic stochasticity, empirical deployments with 16-bit precisions still exhibit catastrophic output divergence across heterogeneous GPUs. Through SASS-level profiling, we reveal that this inconsistency is fundamentally driven by truncation errors introduced during downcasting at kernel boundaries. However, achieving reproducibility via a global FP32 pipeline incurs prohibitive system penalties: bypassing 16-bit hardware accelerators hurts compute efficiency, while upca…

2. Related Groundwork for this paper Cited by this paper Not relevant

   Bit-Accurate Modeling of GPU Matrix Multiply-Accumulate Units: Demystifying Numerical Discrepancy and Accuracy

   Peichen Xie, Shuotao Xu, Yang Wang et al. · 2025 · cited 4 times · Not summarized yet

   This paper models GPU matrix multiply-accumulate arithmetic to explain numerical discrepancies, providing a modeling precedent for the Evo 2 paper’s arithmetic investigation.

   Abstract

   Modern AI accelerators rely on matrix multiply-accumulate units (MMAUs), such as NVIDIA Tensor Cores and AMD Matrix Cores, to accelerate deep neural network workloads. MMAUs expose only instruction-level or API-level interfaces of matrix multiply-accumulate (MMA) operations, while leaving internal floating-point arithmetic behavior undocumented. Consequently, MMAUs across vendors and architectural generations often produce numerical discrepancies for identical inputs, and sometimes exhibit reduced numerical accuracy that can cause training instability. Diagnosing and understanding the root causes of these effects is challenging without white-box models of their arithmetic behavior. This pape…

3. Related Broader context Not relevant

   CurveFP: Co-Designing Numerical Representation and Product Arithmetic for Language Models

   Ye Qiao · 2026 · Not summarized yet

   CurveFP redesigns low-precision representations and product arithmetic for language models, whereas this paper diagnoses compatibility failures in fixed FP8 checkpoints.

   Abstract

   Low-precision formats usually optimize scalar fidelity while inheriting conventional product arithmetic. We introduce CurveFP, a block-scaled family that distributes magnitudes across interleaved logarithmic curves. Uniform curve indices make every nonzero product an exact sign and integer-index update, while a rational radix exposes the finite phase schedule required for accumulation. We instantiate the algebra as CurveFP8 E4C3/E5C2 for training and CurveFP7 E3C3 for compact inference. On four 7B-9B models, CurveFP7 beats tensorwise FP8 perplexity with one fewer element bit and stays within 1.32% of native quality. CurveFP8 lowers error in all 36 paired training-GEMM comparisons. Across thr…

4. Related Broader context Not relevant

   Spec Sheets Are Not Kernels: An ISA- and Source-Level Audit of INT8 Availability on NVIDIA Blackwell Ultra

   Teng-Ruei Chen · 2026 · cited 2 times · Not summarized yet

   This audit examines Blackwell’s INT8 hardware and software support, while the Evo 2 paper tests how Blackwell FP8 arithmetic affects model compatibility.

   Abstract

   NVIDIA's published specifications give the Blackwell Ultra GPU (B300) a dense-compute ratio of roughly 30:1 between FP8 and INT8 tensor-core throughput; its predecessors, H200 and B200, both provide 1:1. We audit what this deprioritization means in practice by tracing INT8 W8A8 support through four layers of the stack: the published specifications, the PTX ISA, NVIDIA's CUTLASS kernel library, and the two major open-source LLM serving engines (vLLM and SGLang). We find a consistent, layered withdrawal: (i) the PTX ISA never exposes the fifth-generation tensor-core integer path (tcgen05.mma with .kind::i8) on sm_103a, even though the same PTX revision extends the FP4 kinds to that target, lea…

5. Related Shared data, tool or method Not relevant

   Hawkeye: Reproducing GPU-Level Non-Determinism

   Erez Badash, Dan Boneh, Ilan Komargodski et al. · 2026 · arXiv.org · preprint, not peer reviewed · cited 3 times · Not summarized yet

   Hawkeye reproduces GPU matrix-multiplication operations on CPUs, sharing the Evo 2 paper’s focus on reproducing GPU arithmetic behavior.

   Abstract

   We present Hawkeye, a system for analyzing and reproducing GPU-level arithmetic operations. Using our framework, anyone can re-execute on a CPU the exact matrix multiplication operations underlying a machine learning model training or inference workflow that was executed on an NVIDIA GPU, without any precision loss. This is in stark contrast to prior approaches to verifiable machine learning, which either introduce significant computation overhead to the original model owner, or suffer from non-robustness and quality degradation. The main technical contribution of Hawkeye is a systematic sequence of carefully crafted tests that study rounding direction, subnormal number handling, and order o…

6. Related Broader context Not relevant

   Reference Traces for Auditing Invisible Weight Updates and Guiding Exact-Budget Protection

   Zekai Shang · 2026 · arXiv.org · preprint, not peer reviewed · Not summarized yet

   This paper audits low-precision optimizer write-back against high-precision traces, rather than inference-time FP8 accumulation in fixed checkpoints.

   Abstract

   Direct low-precision write-back can erase nonzero optimizer proposals. We ask what a high-precision reference trace establishes before a low-precision run. The exact target-code event is auditable coordinatewise on a realized target trajectory; pre-run aggregate projection also assumes the reference remains a useful counterfactual. In a controlled two-layer grid, 55/72 cells have measured and predicted post-initialization crossings: times span $384\times$, 52/55 are within 15\%, and 4/72 differ in category. Matched decoder experiments show stochastic rather than nearest write-back recovers most of the loss gap. A prospective analytic-grid E4M3 audit reuses one fp32 trace across three unseen …

7. Related Broader context Not relevant

   Bridging Consumer and Enterprise: Benchmarking LLaMA, Qwen, and GPT Models on Diverse Hardware

   F. Khan, Shoaib Mustafa Khan · 2026 · International Conference on Multimodal Interaction · Not summarized yet

   This benchmark compares LLM performance across hardware platforms, while the Evo 2 paper studies checkpoint accuracy and compatibility across GPUs.

   Abstract

   The increase of large language models (LLMs) such as Qwen, LLaMA, Mistral, and GPT variants has intensified the need for systematic benchmarking across heterogeneous hardware platforms. This paper evaluates NVIDIA A100 and RTX 3090 GPUs, AMD RX 580 and RX 7600 XT GPUs, and an Intel i5-14400F CPU on a diverse suite of open-source LLMs, including Mistral-7B, LLaMA-2 (7B, 13B), LLaMA-3.x (3B–11B), CodeLLaMA, Qwen1.5/2/2.5 (1.8B–14B), and legacy GPT models (GPT-2, GPT-Neo, GPT-J, GPT-NeoX-20B). Performance was assessed in terms of sustained TFLOPs, throughput in tokens per second (TPS), and memory/power efficiency. Results show that smaller Qwen models peaked at 276 TPS (A100), the RTX 3090 domi…

8. Related Broader context Not relevant

   DeepGEMM: Accelerated Ultra Low-Precision Inference on CPU Architectures using Lookup Tables

   Darshan C. Ganji, Saad Ashfaq, Ehsan Saboori et al. · 2023 · Machine Learning · cited 12 times · AI summary available

   DeepGEMM accelerates ultra-low-precision inference on CPUs using lookup tables, unlike the Evo 2 paper’s study of FP8 arithmetic on GPUs.

   Abstract

   A lot of recent progress has been made in ultra low-bit quantization, promising significant improvements in latency, memory footprint and energy consumption on edge devices. Quantization methods such as Learned Step Size Quantization can achieve model accuracy that is comparable to full-precision floating-point baselines even with sub-byte quantization. However, it is extremely challenging to deploy these ultra low-bit quantized models on mainstream CPU devices because commodity SIMD (Single Instruction, Multiple Data) hardware typically supports no less than 8-bit precision. To overcome this limitation, we propose DeepGEMM, a lookup table based approach for the execution of ultra low-precis…

Prior work it builds on

1. Related Groundwork for this paper Not relevant

   Mixed Precision Block Fused Multiply-Add: Error Analysis and Application to GPU Tensor Cores

   Pierre Blanchard, N. Higham, Florent Lopez et al. · 2020 · SIAM Journal on Scientific Computing · cited 71 times · Not summarized yet

   This work analyzes errors in mixed-precision fused multiply-add on GPU tensor cores, relevant to the Evo 2 paper’s investigation of accumulation arithmetic.

   Abstract

   Computing units that carry out a fused multiply-add (FMA) operation with matrix arguments, referred to as tensor units by some vendors, have great potential for use in scientific computing. However...

2. Related Groundwork for this paper Not relevant

   Accurate Models of NVIDIA Tensor Cores

   Faizan A. Khattak, M. Mikaitis · 2025 · ACM Transactions on Architecture and Code Optimization (TACO) · cited 8 times · Not summarized yet

   This work models NVIDIA tensor-core matrix multiplication and its numerical behavior, informing the Evo 2 paper’s analysis of GPU arithmetic differences.

   Abstract

   Matrix multiplication is a fundamental operation in both training of neural networks and inference. To accelerate matrix multiplication, Graphical Processing Units (GPUs) provide it implemented in hardware. Due to the increased throughput over the software-based matrix multiplication, the multipliers are increasingly used outside AI, to accelerate various applications in scientific computing. However, matrix multipliers targeted at AI are at present not compliant with IEEE 754 floating-point arithmetic behaviour, with different vendors offering different numerical features. This leads to non-reproducible results across different generations of GPU architectures, at the matrix multiply-accumu…

3. Related Groundwork for this paper Not relevant

   Numerical behavior of NVIDIA tensor cores

   Massimiliano Fasi, N. Higham, M. Mikaitis et al. · 2021 · PeerJ Computer Science · cited 78 times · Not summarized yet

   This study experimentally characterizes NVIDIA tensor-core arithmetic across GPU generations, groundwork for investigating checkpoint sensitivity to accumulation behavior.

   Abstract

   We explore the floating-point arithmetic implemented in the NVIDIA tensor cores, which are hardware accelerators for mixed-precision matrix multiplication available on the Volta, Turing, and Ampere microarchitectures. Using Volta V100, Turing T4, and Ampere A100 graphics cards, we determine what precision is used for the intermediate results, whether subnormal numbers are supported, what rounding mode is used, in which order the operations underlying the matrix multiplication are performed, and whether partial sums are normalized. These aspects are not documented by NVIDIA, and we gain insight by running carefully designed numerical experiments on these hardware units. Knowing the answers to…

4. Related Groundwork for this paper Not relevant

   Accurate classification of BRCA1 variants with saturation genome editing

   Gregory M. Findlay, R. Daza, B. Martin et al. · 2018 · Nature · cited 792 times · Not summarized yet

   This study provides functional classifications of BRCA1 variants, which the Evo 2 paper uses to evaluate variant-classification performance.

   Abstract

   Variants of uncertain significance fundamentally limit the clinical utility of genetic information. The challenge they pose is epitomized by BRCA1, a tumour suppressor gene in which germline loss-of-function variants predispose women to breast and ovarian cancer. Although BRCA1 has been sequenced in millions of women, the risk associated with most newly observed variants cannot be definitively assigned. Here we use saturation genome editing to assay 96.5% of all possible single-nucleotide variants (SNVs) in 13 exons that encode functionally critical domains of BRCA1. Functional effects for nearly 4,000 SNVs are bimodally distributed and almost perfectly concordant with established assessment…

5. Related Groundwork for this paper Not relevant

   The pitfalls of verifying floating-point computations

   David Monniaux · 2007 · TOPL · cited 247 times · Not summarized yet

   This work explains how floating-point computation semantics vary with implementation choices, a broader basis for the Evo 2 paper’s arithmetic-compatibility analysis.

   Abstract

   Current critical systems often use a lot of floating-point computations, and thus the testing or static analysis of programs containing floating-point operators has become a priority. However, correctly defining the semantics of common implementations of floating-point is tricky, because semantics may change according to many factors beyond source-code level, such as choices made by compilers. We here give concrete examples of problems that can appear and solutions for implementing in analysis software.