CS Paper Radar

Bao: Automatic Region Placement and Memory Allocation for Intermittent Computing

Byeongjee Kang, Jan Hoffmann, Limin Jia, Brandon Lucia, Feras A. Saad

cs.PL ·submitted 2026-10-02 00:29 UTC · v1
top venuetheory

Abstract

Intermittent computing enables batteryless embedded devices to operate in harsh environment, but frequent power failures interrupt program execution and require careful management of state across power cycles. The core challenge is to guarantee both memory consistency and forward progress while minimizing the energy overhead of checkpointing. Recent compile-time approaches identify regions that fit in the energy buffer. At runtime, the device waits and recharges between the regions. However, they still rely on greedy or path-local heuristics that commit to local decisions and can miss globally lower-overhead boundary placements. To address this limitation, we present Bao, a system that jointly finds optimal energy-aware region formation and memory allocation decisions with formal correctness guarantees, by formulating it as a mixed-integer linear program. We prove that any feasible solution guarantees energy safety and implement our approach in LLVM. Our evaluation on 13 benchmarks across 3 capacitor sizes shows that Bao outperforms existing baselines, achieving 10% faster execution and 52% fewer region boundary hits on average compared to the best baseline.

Comment
Accepted to ASPLOS '27. 28 pages, 12 figures, 10 tables
DOI
10.1145/3845814.3850041

Closest papers

Nearest neighbours by meaning across the whole index.

MEMPOWER: Efficient Power Management with Fine-grained Memory Analysis and Modeling for HPC Workloads

cs.DC ·2026-08-21 ·Nanda Velugoti, Joseph Manzano, Andres Marquez, Nathan Tallent, Kyle Hale
published similarity 0.65
Abstract

Managing the energy consumption and power efficiency of parallel applications is a significant issue in both HPC environments and in the cloud. As emerging applications continue to push against the memory wall of modern machines, the growing imbalance between compute and data movement creates new opportunities to intelligently tune CPU power consumption. Unfortunately, existing frequency and voltage scaling techniques do not adequately capture fine-grained changes in memory access behavior, rendering the compute/data access imbalance invisible to the components of the system that could capitalize on it, thus leaving potential power savings on the table. In this paper, we propose MEMPOWER, a flexible, model-based approach to exposing compute/data movement imbalance that characterizes the fine-grained memory behavior of parallel workloads. This characterization then informs our automated software framework which can statically instrument the application binary with model-determined voltage/frequency transitions that balance fine-grained changes in memory access behavior with the costs of hardware transitions. Using MEMPOWER, we demonstrate a reduction in EDP of 6% to 42% on a range of HPC benchmarks with minimal impact on execution time when compared to the standard OS/hardware-managed power control mechanism.

0.33

Energy Aware Optimization of Non-Deterministic Task-Based Intermittent Computing Nodes with Thimble

cs.NI ·2026-11-11 ·John Madden, Morgan Masters, Alec Levy, Steve McGuire, Colleen Josephson
similarity 0.64
Abstract

Intermittent computing nodes run on harvested energy, which they store in capacitors and spend in short bursts of computation. Choosing those capacitors is a trade-off: large capacitors can power large tasks but leak more current, wasting harvested energy, while small capacitors leak less but cannot power large tasks. Recent work, LEACS, showed that selecting capacitors and mapping tasks to them at compile time improves inference throughput. However, LEACS optimizes the mapping for a predetermined workload, whereas sensor nodes often run data-dependent workloads in which the next task depends on the outcome of earlier ones. We present Thimble, an energy storage system that maps a node's tasks to capacitors at runtime. Thimble models the workload as a Markov chain over tasks and assigns each task to a capacitor by weighing capacitor leakage, the task's energy demand and, when it replans, the likelihood of each future task. We compare Thimble against a reimplementation of LEACS, which ranks every static plan by simulation. When the training trace resembles deployment, Thimble's single design-time solve comes within 0.3--4% of LEACS's throughput. When the deployment conditions drift from the characteristics of the training set, LEACS's throughput-focused static plans can fail sharply. On the lowest-power source we consider, LEACS's plan stalls on deployment traces that harvest less than its training trace, and on some never wakes; Thimble avoids this, with 22--29% higher throughput at the reference workload. A sensitivity sweep over 25 workload profiles on that source shows that Thimble with depth replanning leads LEACS by 22--42% throughout, and what run-time remapping adds: replanning at a fixed interval matches or improves on the static map at 21 of 25 points, by up to 7%.

0.31

Physically Partitioned KVCache Format for CPU--GPU Load Balancing in MoE Inference

cs.DC cs.LG ·2026-09-13 ·Enda Yu, Dezun Dong, Xiangke Liao
similarity 0.63
Abstract

Single-GPU long-context inference with Mixture-of-Experts (MoE) models requires spilling the key-value cache (KVCache) to CPU memory. The spilled KV serves two complementary purposes---transferring to the GPU for attention computation, or computing in-place on the CPU---which demand opposing physical states. The optimal split between them varies with workload, yet existing KVCache abstractions offer only storage semantics over a monolithic object of a single physical state, and cannot express dynamic load balancing. We propose InplaceKVCache, the first KVCache abstraction whose format fixes each byte's physical residency at write time, so that the CPU--GPU load balance can be adjusted without moving data after placement. It realizes this as a four-region layout along two dimensions---device affinity and access pattern---turning load balancing into pure scheduling. Built on this abstraction, WriteScope splits CPU--GPU shares along the sequence dimension, and a portable roofline performance model determines the optimal CPU share as sequence length evolves, with online feedback tracking CPU cost drift. On three MoE models (DeepSeek-V2-Lite, Qwen3-30B-A3B, Mixtral-8$\times$7B) with a 32~GB VRAM budget, WriteScope supports end-to-end inference at the 1M-token aggregate scale. In the long-context regime ($\ge$8K), it achieves geometric-mean speedups of $1.5\times$--$2.5\times$ on A100 and $1.4\times$--$1.7\times$ on V100 over four reproduced baselines, while vLLM, SGLang, and KTransformers fail even with a doubled KV budget. A DeepSeek-V4-Flash case study validates composition with native sparse attention.

0.17

ZOCheck: CPU-Shadow Checkpointing for Zeroth-Order LLM Fine-Tuning

cs.DC ·2026-09-23 ·Minqiu Sun, Xin Huang, Luanzheng Guo, Nathan R. Tallent, Kento Sato +1 more
published similarity 0.62
Abstract · Accepted to SC26 (The International Conference for High Performance Computing, Networking, Storage and Analysis), 2026

Zeroth-order (ZO) optimization is an attractive option for memory-efficient LLM fine-tuning, but its fault tolerance remains underexplored. Unlike first-order training, ZO progress can be represented by lightweight seed-and-scalar step logs, yet naive log-only recovery still incurs replay cost that grows with training progress, and shortcut replay does not preserve the executed floating-point trajectory. We present ZOCheck, a fault-tolerant ZO training system that exploits this replayable structure through a CPU shadow process that continuously replays logged updates, materializes consistent recovery images off the GPU critical path, and persists them asynchronously. ZOCheck therefore combines non-blocking checkpointing during training with fast recovery from a near-current state. We also develop a cost model for choosing the snapshot policy under realistic failure rates. Experiments show that ZOCheck reduces checkpoint overhead by up to 219.7x and recovery latency by 1.55x on average compared with asynchronous full-state checkpointing, translating into up to 21.3x lower end-to-end wasted time across the evaluated failure rates, while preserving exact recovery behavior.

0.65

SARIA: Persistence-Aware KV Caching with Coordinated DRAM--SSD Reload for LLM Agents

cs.OS ·2026-10-22 ·Hui Li, Qihui Zhou, Xiao Yan, Shumeng Zhang, Zijie Dai +5 more
similarity 0.62
Abstract · 12 pages, 16 figures

Multi-turn LLM agents usually reuse long context prefixes, making key value (KV) caching crucial for reducing prefill latency. SSDs provide a low-cost extension to limited DRAM capacity but introduce two key challenges. Eagerly persisting all KV cache blocks incurs substantial write traffic and rapidly consumes finite SSD endurance. Meanwhile, the small number of SSDs provisioned in typical GPU servers limits aggregate reload bandwidth, making SSD-resident KV cache much slower to retrieve than that in DRAM and inflating prefill latency. We present \textsc{Saria} (Selective Admission and Reload for Inference Acceleration), a KV caching system that jointly manages DRAM and SSD by exploiting the session-level temporal locality of LLM agents to reduce SSD writes and accelerate cache reload. On the write path, \textsc{Saria} uses eviction and reuse history to selectively persist KV cache blocks that exhibit reuse opportunities beyond their DRAM residency, avoiding unnecessary writes for KV cache already hosted in DRAM. On the read path, \textsc{Saria} promotes selected SSD-hit KV cache blocks from GPU memory into DRAM after prefill, allowing subsequent reuse within the same session to reduce SSD read traffic while limiting DRAM contention. A transfer coordinator paces DRAM reads while SSD transfers are active, allowing both paths to make progress and efficiently share GPU ingress bandwidth. We implement \textsc{Saria} on LMCache MP and evaluate it using workloads from TraceLab and WEKA, covering both full-attention and hybrid-attention models. The results show that \textsc{Saria} reduces SSD writes by 57.1--58.9\% compared with popular full duplicate caching and the mean time to first token (TTFT) by 42.3--79.7\% over DRAM-only caching.

0.08

Execution-Feasible Dispatch of LLM Pre-training Loads for Renewable-Dominant AI Data Centers

cs.DC ·2026-10-09 ·Xi Tong, Ershun Du, Haiyang Jiang, Long Chen, Qin Wang
similarity 0.62
Abstract · 12 Figures, 6 Tables

AI data centers (AIDCs) are emerging as large power-system loads, making their operational flexibility increasingly valuable. Existing studies often treat AIDC demand as continuously adjustable or generically delay-tolerant, which overstating the flexibility of large language model (LLM) pretraining. This paper proposes an execution-feasible dispatch framework for LLM pre-training in renewable-dominant AIDCs. LLM descriptors are first converted into checkpoint-domain daily workloads, then mapped to feasible 3D-parallel GPU modes with explicit throughput, GPU-capacity, memory, and IT-power limits. The resulting checkpoint- and mode-constrained load states are embedded into a day-ahead power dispatch model. Case studies on a 2048 GPUs AIDC with two LLaMA pretraining tasks show that checkpoint-aware 3D dispatch reduces operating cost by 20.02% and renewable curtailment from 1.077 to 0.014 MWh relative to fixed continuous training. Related results also show that checkpoint records alone do not create load flexibility,practical grid-interactive flexibility arises from checkpoint waiting combined with execution-feasible 3D-mode compensation.

0.20

PRIM: Energy- and Endurance-Aware Compiler Optimization for Digital Compute in Non-volatile Memory

cs.ET ·2026-10-12 ·Patrick Miller, Ulya R. Karpuzcu
similarity 0.62
Abstract

Compute-in-Memory (CiM) is an alternative computing paradigm that circumvents the memory bandwidth bottleneck. Variants utilizing non-volatile memory cells (NVCiM) to perform Boolean operations directly within memory arrays offer significant density and energy-efficiency advantages. Fully realizing this potential, however, requires NVCiM-specific compiler optimizations. In conventional digital NVCiM systems, an output cell must be written -- or preset -- to an operation-specific value before any Boolean operation can occur. Preset-induced extra writes can degrade energy efficiency and cell endurance, often to the point of negating CiM's efficiency gains. In this paper, we introduce PRIM (Preset Reduction In Memory), a compiler optimization framework tailored for digital NVCiM that effectively minimizes preset overhead without compromising computational accuracy or architectural complexity. Unlike existing optimization approaches, PRIM is not restricted to specific types of Boolean operations. Experimental results show that PRIM can reduce execution time and energy consumption by up to 50/% and 44/%, respectively, while boosting cell endurance by up to 50/%.

0.14

Ready Cohorts: Bounding GPU Opportunity and Avoiding Host Round Trips in LLM-Agent Control

cs.DC cs.AIcs.OS ·2026-08-12 ·Josef Liyanjun Chen
benchmarktheorycode/data similarity 0.61
Abstract · 14 pages, 4 figures. Includes formal proofs, trace provenance, and a reproducibility appendix. Code and artifacts: https://github.com/josefc…

We introduce a mechanism that improves LLM-agent execution: keeping four-byte, GPU-computed control-path routing decisions on-device, avoiding a round-trip to host memory for redispatch. It is faster than host-dispatch in all 36 placements x mechanisms settings in our benchmark across four named GPUs with row-median speedup of 1.19x-2.39x, and outputs are correct, matching an independent host implementation, for all 14,557,440 calls in all configurations, in both tested mechanisms. Agent control paths can be deterministic. Their transitions between LLM calls and external tool calls can be GPU-accelerated by batching across agents and time. We define a cohort of transitions with launch times in a window, and study four shares of GPU-active time: a fixed-partition baseline F, an optimal P*, an upper bound U, and an online result A. Under assumptions of uniform relative launch deadlines, infinite capacity, and zero service times, we compute P* with a custom dynamic program and validate our simulator against it. We replay an anonymized 851-session trace, shown in full as a panel, stationary-ized with a Poisson process. With 100,000 target active agents and a launch deadline of 50 ms, where the headline results have K=256, we find F=30.19%, P*=43.00%, U=45.85%; our method for computing P* reveals that 81.83% of this opportunity is recoverable from cohort packing at fixed window ends. Our method for U relies on a novel notion of route-key, used only as a proxy (via conditional statistics) and does not purport to establish route-equivalence. We find a nested graph that retains some host-side decisions slower in all 60 configurations (five placements).

0.30