CS Paper Radar

How Numerical Errors Scale with Hardware

Niklas Baumgarten

cs.DC ·submitted 2026-10-26 13:25 UTC · v1

Abstract

In this short technical note, we outline how to bound numerical errors with respect to computational resources. We consider algorithms with known complexity estimates that can be executed in parallel and combine these estimates with parallelization laws. We suppose that the proposed approach generalizes beyond the presented examples and argue that it provides a straightforward and fair framework for comparing algorithms, hardware systems, and software implementations. This is achieved by imposing identical resource or energy budgets on each configuration, and comparing the resulting numerical error evaluations.

Closest papers

Nearest neighbours by meaning across the whole index.

Running error bounds in finite element kernels

cs.MS math.NA ·2026-09-29 ·Michal Habera, Paul T. Kühner, Matteo Croci, Andreas Zilian
code/data similarity 0.58
Abstract · 18 pages

Rounding errors in finite element computations can lead to a complete loss of accuracy, stalled convergence, and incorrect results. Moreover, the effects of rounding errors accumulated within automatically generated and compiled kernels are difficult to analyze a priori. We present the first software framework for automated rounding error estimation within finite element kernels. The proposed methodology is based on an a posteriori technique called Running Error Analysis (REA), where a forward error estimate is automatically computed concurrently with the value. An open-source implementation is provided for the FEniCS Form Compiler (FFCx), based on a C++ backend for generating type-generic templated kernels over a custom arithmetic type that tracks both the value and its error estimate. We demonstrate REA on two examples. First, we use it to detect catastrophic cancellation in the assembly of a Neo-Hooke hyperelastic model in the small deformation regime. A series expansion circumvents the cancellation problem and the computed error estimates show this. Second, we study the assembly of the Laplace operator on a near-degenerate mesh. We demonstrate that our REA implementation typically incurs only 2-4x performance overhead. Applications of this work include robust reduced-precision computations in embedded systems, numerical debugging of new, possibly ill-conditioned or unstable PDE formulations, and guiding the design of mixed-precision kernels.

0.40

Lord of the Precision, the Two Floats: Double-Word Arithmetic for Complex Numbers

cs.MS ·2026-11-05 ·František Stloukal, Fabienne Jézéquel
code/data similarity 0.58
Abstract · 46 pages, 8 figures, 3 tables

Datacenter GPUs increasingly favour low-precision floating-point formats, offering higher throughput than FP64. Double-word (DW) arithmetic represents numbers as unevaluated sums of two floating-point values, providing higher accuracy. We present the first error analysis of complex DW multiplication and division. Both operations reduce to sums of two DW products, combined using one of three DW addition algorithms: accurate, sloppy, or Madd. Normalising DW products ensures nonoverlapping components, but omitting this step reduces the number of arithmetic operations. We bound the resulting overlap and prove that Madd retains a relative error bound of $3u^2+O(u^3)$ for overlapping inputs, where $u$ is the unit roundoff. For all variants, we establish relative error bounds of the form $(ακ+β)u^2+O(u^3)$, where $κ$ is the condition number of the sum of products. Our analysis shows that the less expensive sloppy addition suffices. We implement these variants in CompleXDW, an open-source C++ library for CPUs and GPUs. In the high-energy physics code MadGraph7, FP32-based DW arithmetic achieves up to 2.9x and 2.1x higher throughput than FP64 on NVIDIA L40S and AMD Radeon Pro W7900 GPUs, respectively, while losing approximately 1.4 correct significant digits.

0.39

Mixed-Precision Computing for Scientific Discovery: Formats, Co-Design, and Responsible Approximation

cs.AR math.NAcs.MS ·2026-09-29 ·Emmanuel Agullo, Hartwig Anzt, Daniel Bauer, David Bindel, Alfredo Buttari +35 more
similarity 0.57
Abstract · 41 pages

Reduced and mixed precision have moved from a niche optimization to a central design axis in scientific computing and engineering, driven by energy constraints, heterogeneous accelerators, and the convergence of simulation and machine learning. This paper organizes the landscape around seven coupled themes---number formats, floating-point emulation, emerging architectures, hardware/software co-design, relation to other approximations, software design, and precision as a multilevel resource ---and, for each theme, synthesizes the state of the art, future directions, and open questions. We emphasize \emph{energy per trusted solution} as the core objective, and we frame \say{recklessly responsible} computing as a pragmatic doctrine: exploit low precision aggressively, but with systematic detection, escalation, and certification pathways.

0.52

What Bloats Your Floats? Right-Sizing Numerical Precision for Scientific Computing

cs.DC ·2026-10-06 ·Pratyai Mazumder, Alexandru Calotoiu, Marcin Copik, Yakup Koray Budanaz, Shriram Chandran +1 more
published similarity 0.56
Abstract · Accepted at SC '26: The International Conference for High Performance Computing, Networking, Storage, and Analysis, November 15-20, 2026, Ch…

While FP64 (binary64) remains the standard representation of real numbers in scientific computing, hardware trends increasingly favor low-precision formats optimized for AI workloads. This shift prioritizes throughput and energy efficiency over full-precision capabilities. We challenge the necessity of high-precision arithmetic in PDE-governed simulations, where spatial and temporal discretization errors introduce a physical noise floor that masks the lower-order bits of the FP64 format, rendering them irrelevant. We propose a criterion based on the Signal-to-Noise Ratio (SNR) with respect to spatial and temporal refinements to evaluate the impact of safe precision reduction on scientific computations. Finally, we present an automated dataflow-centric workflow for targeted lowering within mixed-precision subgraphs in complex scientific applications. Large-scale evaluations of weather and climate applications show that strategic precision reduction on GPUs achieves up to 1.8x speedup without compromising physical fidelity.

0.36

Adaptation Fidelity of SPEC CPU2026

cs.PF ·2026-08-27 ·Doa'a Al-Otoom, Mahesh Madhav
publishedcode/data similarity 0.55
Abstract · 5 pages, 2 figures, Presented at the 2026 IEEE International Symposium on Workload Characterization (IISWC 2026), Boulder, Colorado

Standardized benchmarks are often criticized for not being "real workloads," but this critique is rarely backed by data. This paper provides the first systematic, quantitative analysis of the "fidelity gap" between the SPEC CPU2026 suite and its original, upstream open-source counterparts. We compile both the SPEC benchmarks and their upstream applications and execute them with official input workloads under two scenarios: a single-copy latency run and a 192-copy throughput run. Our findings show that most benchmarks exhibit high fidelity in single-copy runs, while a few outliers reveal the impact of SPEC's adaptation process. The multi-copy results further highlight the necessity of this adaptation: several benchmarks become significantly more efficient than their upstream versions under heavy load, underscoring the importance of I/O reduction. This work offers data-driven validation of SPEC's methodology, showing that the fidelity gap is not a flaw but a quantifiable consequence of enforcing portability, determinism, and CPU-centric measurement.

0.41

Adaptive Parallel-in-Time Integration with Dynamic Resource Management

cs.CE ·2026-09-21 ·Keerthi Gaddameedi, Dominik Huber, Martin Schreiber, Hans-Joachim Bungartz, Valentina Schüller +2 more
similarity 0.54
Abstract

As computational resources continue to grow, the strong-scaling limitations of spatial parallelism motivate the pursuit of additional concurrency in the temporal dimension, particularly for applications with hard time constraints, such as weather and climate simulations. The Parallel Full Approximation Scheme in Space and Time (PFASST) is a parallel-in-time method based on Spectral Deferred Corrections (SDC). It computes multiple timesteps concurrently by coupling fine- and coarse-grid SDC sweeps using multigrid Full Approximation Scheme (FAS) corrections. However, PFASST's convergence is often problem-dependent, demanding a variable number of parallel timesteps and, hence, computing resources at different times throughout the simulation. Dynamic Resource Management (DRM) provides a remedy for this challenge by enabling the adaptive adjustment of computational resources and algorithmic parameters at runtime. In this work, we present our novel approach to extending PFASST with DRM, which enables (a) dynamic adaptation of computing resources, (b) adaptive selection of the number of PFASST iterations based on local convergence behavior, and (c) coupling of these two adaptations into a single resizing strategy. With this approach, we demonstrate for the first time that optimal configurations can be identified in real time for each application, rather than relying on static allocation. Furthermore, we show that convergence-informed tuning of PFASST improves resource utilization and convergence efficiency.

0.28

Providing Accuracy Guarantees for the Fixed-Point Emulation of Matrix Multiplication

cs.MS cs.DC ·2026-11-13 ·Cole Brower, Peter Caday, John Gunnels
theory similarity 0.54
Abstract · We plan to submit to TOMS, but have not done so

Fixed-point methods such as Ozaki-II emulate double-precision matrix multiplication (GEMM) on fast low-precision hardware. Software that chooses the mantissa bit count automatically is numerically safe when the emulated product keeps the error guarantee of the floating-point GEMM it replaces. We analyze the worst-case and average-case error of fixed-point and floating-point dot products, define exponent span capacities that determine the extra mantissa bits an input requires, and approximate them with a low-precision GEMM. We prove that 53 bits plus such a capacity and a small constant preserve the classical worst-case error bound for a floating-point dot product, and we demonstrate this guarantee in software on several test cases. In a developer build of cuBLAS on an NVIDIA B200, each implemented bit-count rule earns grade~A on Test~2c of ``Grading the BLAS'' at every size and exponent range tested, falling back to double precision at the widest ranges; in exact arithmetic, the bound holds on every adversarial and random input tested.

0.32

Closing the Prediction Gap: Completing Machine Shape So That Predicted Time, Power, Energy, and Mapping Match What Real Hardware Does

cs.DC ·2026-10-02 ·Lenore Mullin, Gaetan Hains
similarity 0.54
Abstract

A companion paper derives a weighted, communication free partition for heterogeneous, multi institution device ensembles from each device measured shape, rho_machine(d), validated on real hardware. Its experiments show rho_machine is incomplete: fp32 to fp16 speedup and power shifts are not fully indexed, and memory capacity predictions rely on an assumed, not measured, reservation overhead. This paper closes that gap, indexing five unmeasured fields in the psi selection vocabulary: memory capacity $M_i^{cap}$, power draw $P_i$, execution unit capability $X_d$, compiler directive configuration $C_d$, and memory reservation overhead $R_d$. We introduce reached(d), a Boolean for whether compiled code dispatches to specialized hardware, explaining the NVIDIA fp16 power shift that hardware availability alone cannot predict. MoA fixes a computation independently of its target, so compiler directives get a two part admissibility criterion: select a real hardware quantity without discretionary reordering, and be deterministic, not compiler overridable. Across Intel, AMD, NVIDIA, OpenMP, OpenACC, and Open MPI, this admits only cache placement controls, deterministic vector or tile widths, and occupancy caps; heuristics like autotuning are excluded and absorbed into a residual epsilon$(k)_d$. We validate on five devices: A100, H100, V100, MI100, and Max 1550, measuring or establishing all five fields on each. Reachability proves real but partial, occupancy limits are device specific, and reservation overhead ranges about 0.004 to 0.030. A residual of negative 9.8 percent for the excluded Triton autotuner shows an unvalidated heuristic can act opposite a presumed penalty. These results close all five fields for every device in the companion ensemble, converting assumed behavior into measured, target specific quantities.

0.29