CS Paper Radar

Adaptive Parallel-in-Time Integration with Dynamic Resource Management

Keerthi Gaddameedi, Dominik Huber, Martin Schreiber, Hans-Joachim Bungartz, Valentina Schüller, Tobias Neckel, Martin Schulz

cs.CE ·submitted 2026-09-21 11:29 UTC · v2

Abstract

As computational resources continue to grow, the strong-scaling limitations of spatial parallelism motivate the pursuit of additional concurrency in the temporal dimension, particularly for applications with hard time constraints, such as weather and climate simulations. The Parallel Full Approximation Scheme in Space and Time (PFASST) is a parallel-in-time method based on Spectral Deferred Corrections (SDC). It computes multiple timesteps concurrently by coupling fine- and coarse-grid SDC sweeps using multigrid Full Approximation Scheme (FAS) corrections. However, PFASST's convergence is often problem-dependent, demanding a variable number of parallel timesteps and, hence, computing resources at different times throughout the simulation. Dynamic Resource Management (DRM) provides a remedy for this challenge by enabling the adaptive adjustment of computational resources and algorithmic parameters at runtime. In this work, we present our novel approach to extending PFASST with DRM, which enables (a) dynamic adaptation of computing resources, (b) adaptive selection of the number of PFASST iterations based on local convergence behavior, and (c) coupling of these two adaptations into a single resizing strategy. With this approach, we demonstrate for the first time that optimal configurations can be identified in real time for each application, rather than relying on static allocation. Furthermore, we show that convergence-informed tuning of PFASST improves resource utilization and convergence efficiency.

Closest papers

Nearest neighbours by meaning across the whole index.

Extending Grid Foundation Models for Phasor-Domain Dynamic Simulation

eess.SY cs.LG ·2026-11-06 ·Boyang Zhao, Yifan Zhou, Alban Puech, Tilman Bockhacker, Hendrik Hamann
similarity 0.62
Abstract

Grid Foundation Models (GridFMs) are emerging as a powerful framework for learning and transferring knowledge across power grids and grid analysis tasks. This work extends GridFMs from steady-state analysis to dynamic simulation. We present DynGridFM, which uses component-wise tokenization and a structured latent operator to learn shared dynamic representations across heterogeneous grids and enable transfer to unseen systems. Pretrained on nine grids, DynGridFM is adapted to a held-out 48-machine grid, where fine-tuning on 64 trajectories achieves accuracy comparable to training from scratch on 128. On a single GPU, it achieves 924x to 1232x speedup over numerical simulators on one CPU core.

0.26

Space-Time Parallel Direct Numerical Simulation of Rayleigh-Bénard Convection

cs.DC math.NAphysics.flu-dyn ·2026-10-13 ·Thomas Saupe, Thibaut Lunet, Mathis Bode, Robert Speck, Sebastian Götschel +1 more
similarity 0.60
Abstract

Rayleigh-Bénard convection (RBC) is a model for turbulent flow and challenging to simulate by direct numerical simulation (DNS). Adequate resolution of all relevant length scales requires very fine grids in space and stability restrictions can dictate small step sizes. In this paper, we investigate the use of parallel spectral deferred correction (SDC) as a high-order time-stepping method to reduce wall-clock times of direct numerical simulation of RBC. Performance results on JUSUF (CPUs) and JUPITER (GPUs) with two different codes show parallel SDC can offer some concurrency along the temporal axis plus larger maximum stable time steps. Taken together, this allows for a reduction of wall-clock time required for simulations over one free-fall time by having to compute fewer steps and being able to mitigate the higher cost per time step through temporal parallelization.

0.24

Online Autotuning of Algebraic Multigrid via Contextual Bandits and Reinforcement Learning

cs.LG math.NA ·2026-10-31 ·Xudong He, Ruiyu Wang, Yilang Ding, Rui Peng Li, Yuanzhe Xi
benchmarkcode/data similarity 0.56
Abstract · 26 pages, 7 figures, 7 tables. Code: https://github.com/EriseHe/BoomerAMG-Autotuner

Algebraic multigrid (AMG) methods are widely used for solving large sparse linear systems, but their efficiency depends strongly on interacting setup and solve parameters. Effective parameter choices typically depend on the problem. We develop a phase-aware online autotuning method for hypre's BoomerAMG to reduce cumulative time-to-solution over sequences of related linear systems. A LinUCB-based contextual bandit selects mixed-type parameters for AMG hierarchy construction using a shared cost model over PDE descriptors and configuration encodings. A recursive LSTDQ controller uses residual and timing observations to estimate the AMG solve time within each attempt and adapt relaxation weights between multigrid cycles. Both learners update from ongoing solves without offline pretraining. Across grids of sizes $40^3$, $60^3$, and $80^3$, the joint autotuning method reduces total runtime, including recurring learner computation, by 36.1-43.3% for diffusion problems and 19.8-31.0% for diffusion-advection problems over sequences of 5000 linear systems, relative to the reference BoomerAMG default configuration. On a variable-flow diffusion-advection benchmark comprising 30,000 systems on a $60^3$ grid, 2 systems fail to converge with joint autotuning, compared with 1677 for the reference configuration, under their respective attempt budgets.

0.36

ParaAnya: Accelerating Parallel Diffusion Sampling with Plug-and-Play Output Caching

cs.DC ·2026-09-29 ·Chee-En Yu, Xiao-Xi Tan, Yi-Cheng Lin, Yun-Shao Tsai, Chee-An Yu +1 more
similarity 0.55
Abstract · 5 pages

Diffusion models have achieved remarkable success in generative tasks, but their inherently sequential sampling process introduces a severe computational bottleneck. Recent Parallel-in-Time (PinT) solvers attempt to mitigate this by parallelizing generation across a sliding window of timesteps, advancing the window only when step-wise changes stabilize. However, this overlapping window mechanism forces the network to repeatedly evaluate the same timesteps. When the input variations between iterations are minimal, these redundant evaluations lead to significant computational waste. To address this inefficiency, we propose ParaAnya, an output cache mechanism agnostic to the parallel sampling algorithm that can reduce the number of function evaluations (NFE). ParaAnya caches input-output pairs of diffusion models and reuses the cached output at overlapping timesteps. By dispatching only cache-miss timesteps to GPU workers, our approach eliminates redundant computation while preserving the structure of the underlying algorithms' update rules. We integrate ParaAnya into four representative parallel sampling algorithms and evaluate its performance on Stable Diffusion v1.5. Across four parallel samplers evaluated with DDIM on eight GPUs, ParaAnya provides $1.30$--$2.43\times$ speedups over their uncached counterparts and reduces NFE by up to 70.1\%, reaching up to a $5.62\times$ speedup over single-GPU serial sampling while maintaining comparable CLIP scores.

0.49

SPMW: A Grid-Based Programming Model for Scalable Accelerator Design

cs.PL cs.AR ·2026-11-06 ·Hongzheng Chen, Shihan Fang, Niansong Zhang, Sunwoo Kim, Weiran Liu +1 more
similarity 0.54
Abstract

Modern accelerators specialize recurring computation and replicate it across regular hardware structures to achieve high performance. Sustaining this parallelism requires equally deliberate data delivery. Yet design tools often express these structures as sequential programs or flatten them into separate hardware modules and connections. This obscures shared patterns and leads to repeated compiler work. We introduce Single Program, Multiple Connected Work Units (SPMW), a grid-based programming model for scalable accelerator design that makes grid regularity a first-class abstraction. SPMW decouples local work-unit behavior from grid connectivity and data delivery. To our knowledge, SPMW is the first programming model to retain regularity throughout the compilation process, combining behavioral HLS with structural RTL generation for both fixed-function and domain-specific instruction-programmable accelerators. We implement SPMW with a Python frontend and an MLIR-based compiler that synthesizes each distinct kernel configuration once for reuse across independent hardware instances. Preserving regularity also enables parallel compilation and functional simulation to accelerate iterative design feedback. We use SPMW to generate a 32x32 systolic array that reduces GEMM latency and parallel kernel-to-RTL compilation time by 4.36x and 619x relative to AutoSA. Our FFT achieves up to 2.65x lower latency than HP-FFT. Our mini-TPU and our SPMW implementation of the FEATHER accelerator demonstrate complex composition and outperform their RTL baselines. Parallel functional simulation provides up to 18.4x speedup over Vitis HLS C simulation on the mini-TPU workloads. SPMW is also agent-friendly: given the same design specification, all five coding agents produce SPMW designs that satisfy the correctness and latency requirements, with 7.51x lower token usage than for SystemVerilog.

0.19

Mixed-precision GPU algorithms for efficient turbulent flow simulations with Raviart-Thomas finite elements

cs.MS physics.flu-dynmath.NA ·2026-09-17 ·Ivan Prusak, Enes Mustafa Soydan, Ivan Pribec, Martin Kronbichler
similarity 0.54
Abstract · 19 pages, 9 figures, 5 tables

We propose GPU algorithms for high-fidelity simulation of incompressible turbulent flows. Discretization in space is performed with H(div)-conforming high-order Raviart-Thomas finite elements for the velocity and an $L^2$-conforming discontinuous Galerkin approximation for the pressure. In time, a consistent splitting scheme based on higher-order BDF time stepping is used, with convection treated explicitly. In this scheme, a pressure Poisson equation and a symmetric reaction-diffusion-type equation for the velocity need to be solved in each time step. We develop a solution framework with fast matrix-free operator evaluation for all ingredients, combined with multigrid solvers for the Poisson problem, and propose a robust mixed-precision algorithmic framework. A key to mixed-precision efficiency is a least-squares projection to generate accurate initial guesses for the iterative linear solvers, enabling us to work with relative residual tolerances of $10^{-3}$. In this regime, running the solvers entirely in single precision leads to almost no change in overall iteration counts and maintains the crucial turbulence statistics, while showing up to $1.7\times$ speedup over pure double-precision simulations.

0.52

How Numerical Errors Scale with Hardware

cs.DC ·2026-10-26 ·Niklas Baumgarten
similarity 0.54
Abstract

In this short technical note, we outline how to bound numerical errors with respect to computational resources. We consider algorithms with known complexity estimates that can be executed in parallel and combine these estimates with parallelization laws. We suppose that the proposed approach generalizes beyond the presented examples and argue that it provides a straightforward and fair framework for comparing algorithms, hardware systems, and software implementations. This is achieved by imposing identical resource or energy budgets on each configuration, and comparing the resulting numerical error evaluations.

0.24

Mixed-Precision SEM-Based CFD Simulations on GPUs: A Taylor-Green Vortex case

cs.DC cs.SE ·2026-08-25 ·Yanxiang Chen, Manuel Münsch, Roman Iakymchuk
similarity 0.54
Abstract

Mixed precision is a promising approach for reducing the computational cost and energy consumption of Computation Fluid Dynamics (CFD) simulations, but its effectiveness depends strongly on where precision is reduced within the full simulation pipeline. In this work, we study Taylor-Green vortex case using Neko, a matrix-free CFD solver based on the spectral element method (SEM). Profiling shows that the fluid time step is not dominated by Krylov convergence alone: the velocity and pressure solvers require only a small number of iterations per step, while a substantial fraction of runtime is spent in other SEM operators and solver components. Motivated by this structure, we propose a three-level hierarchical mixed-precision control model. Two groups of configurations are evaluated in environments bounded by 64-bit floating-point (fp64) and 32-bit floating-point (fp32) precision, respectively. The fp64-bounded group identifies accuracy sensitive components and shows that SEM-focused fp32 computation is a promising direction for future optimization. The fp32-bounded group provides the main practical benefit. For the high Reynolds number case studied, selected configurations reduce both time- and energy-to-solution by about 34% relative to the fp64 baseline, while improving robustness compared with global fp32. Targeted fp16 kernel overrides are also explored, showing potential for selected operations but increased sensitivity in gradient-based quantities such as enstrophy. Overall, these results indicate that mixed-precision for matrix-free SEM-based CFD should be treated as a simulation-level control problem rather than solely as a Krylov-solver optimization.

0.26