CS Paper Radar

Dynamic Slack-Aware Clocking for Near-Threshold Tensor Processing Units (TPUs)

Muhammad Usman Nadeem, Sanghamitra Roy, Koushik Chakraborty

cs.AR ·submitted 2026-09-22 16:16 UTC · v1
top venue

Abstract

Operating Tensor Processing Units (TPUs) in the near-threshold computing (NTC) region significantly reduces energy consumption but introduces high delay sensitivity to process variation and data activity. Conventional designs typically rely on a conservative, fixed global clock to ensure safety, which leaves large portions of timing margin unexploited as most operations finish well before the clock edge. We propose Dynamic Slack-Aware Clocking (DSAC), a proactive framework that replaces worst-case timing with operation-specific adjustments. DSAC employs lightweight Hamming-Distance, Most-Significant-Bit, and Hybrid predictors to estimate the delay sensitivity of individual multiply-accumulate (MAC) operations and classify them into three timing tiers. These tiers are enforced locally via dummy-hold cycles under a fixed global reference clock, enabling fine-grained timing adaptation without global clock retuning or frequency scaling. A closed-loop feedback controller monitors timing violations and updates tier thresholds at runtime to maintain resilience. Experiments on quantized DNN benchmarks demonstrate that the MSB predictor maintains high inference accuracy, with an average loss of only 1% even at aggressive performance points. Furthermore, DSAC achieves up to 1.55X better energy efficiency at 2.15X frequency scaling compared to a baseline TPU, while incurring an area overhead as low as 13%.

Comment
31 pages, 13 figures. Accepted manuscript of ACM Transactions on Design Automation of Electronic Systems (TODAES); DOI: 10.1145/3837085
DOI
10.1145/3837085

Closest papers

Nearest neighbours by meaning across the whole index.

Context-Aware Precision Scheduling for Small Language Models

cs.LG ·2026-10-11 ·Mukund Athmaram, Manoj Rao, Liang Ma
benchmarkposition similarity 0.71
Abstract

Small language models (SLMs) used as on-device agents must balance accuracy and latency as their context grows. Mixed precision helps manage this tradeoff by assigning different precisions to weight tensors according to their size and sensitivity. However, these choices depend on the operating conditions, so a configuration selected for a short context may no longer provide the best tradeoff at a longer one. We calibrate a timing model from measurements at different precisions and context lengths, and combine it with tensor sensitivities to choose configurations under a latency or accuracy budget. We argue that fewer bits do not always mean faster execution, so our predictor estimates latency from calibrated execution costs rather than weight bytes alone. Under a latency budget, reading the larger key-value (KV) cache leaves less time for weight operations, leading the allocator to select lower precisions. This gives a schedule of tensor changes that can be prepared before execution. Our runtime follows the schedule while retaining the session's KV cache. We evaluate executed schedules on a laptop GPU, including a longer-context schedule with three weight formats and a tool-use benchmark measuring full response time and correctness. Under an accuracy budget, measured error from a quantized KV cache determines the budget left for weights. The allocator can raise or lower weight precision as this estimate changes. These experiments show where context-aware precision helps and where accuracy recovery remains limited.

0.18

Adapting DNNs to Embrace Voltage Scaling Induced Errors via Accumulation-Aware Quantization

cs.AR cs.AI ·2026-10-24 ·Alexis Maras, Panagiotis Chaidos, Georgios Alexandris, Dimitrios Soudris, Sotirios Xydis
published similarity 0.64
Abstract · Accepted for publication at the 45rd International Conference on Computer-Aided Design (ICCAD '26), November 08-12, 2026, San Jose, CA, USA

The pervasive deployment of Deep Neural Networks (DNNs) across the computing spectrum has brought growing attention to energy efficiency, alongside raw performance. While voltage scaling is a potent knob for power reduction, its application in next-generation nodes is hampered by severe throughput degradation and susceptibility to process variation. This work aims to address these energy-reliability trade-offs by introducing a hardware-software co-design methodology, based on the observation that timing-robust bit ranges within the accumulator of a systolic array vary significantly across different voltage-frequency operating points. At the software level, we propose an Accumulation-Aware Training approach that incorporates this information as a first-order optimization constraint during training. By integrating mixed-precision quantization with unstructured pruning, we effectively constrain the model's weight distribution to avoid activating the hardware's sensitized high-rank carry chains. At the architectural level, we complement this with selective MSB-masking and fine-grained clock-gating to minimize unnecessary dynamic switching activity. Furthermore, we employ a layer-wise Dynamic Frequency Scaling strategy to further adapt to the accumulator's reduced dynamic range, based on the numerical sensitivity of each layer. We experimentally evaluate the proposed solution on a deep scaled 2nm technology node and demonstrate that, when operating at a low-voltage frontier, our co-optimized post-layout architecture delivers substantial energy and energy-delay product gains of 39% and 23% respectively, relative to the nominal voltage implementation, while maintaining accuracy degradation within 1%.

0.33

EdgePace: Deadline- and Freshness-Aware LLM Scheduling on Unified-Memory Platforms

cs.PF ·2026-10-27 ·Teng Mei, Chengxuan Pei, Weijun Wang, Jie Wu, Marco Canini +1 more
similarity 0.64
Abstract · 15 pages, including the reference and some interesting finds in a four-page appendix

Large language models (LLMs) increasingly run on edge devices alongside tasks that continuously process sensor data. In many applications, these foreground tasks prepare observations as inputs to the LLM, whose outputs guide subsequent processing or actions. The observations must be processed by foreground deadlines, and LLM feedback must arrive while its source observations remain useful. Yet the CPU and accelerators share a unified-memory system, so LLM inference can delay foreground processing through memory-bandwidth contention even when compute resources remain available. We present EdgePace, a bandwidth-aware runtime that protects foreground deadlines while sustaining timely LLM feedback. EdgePace creates safe suspension points across prefill and decode, monitors foreground memory demand and stalls at low cost, and schedules inference according to contention risk, deadline slack, and output freshness. We evaluate EdgePace on a OnePlus 13 and three NVIDIA Jetson platforms using multi-camera processing and robotic target tracking. Relative to unmanaged inference, it reduces foreground deadline misses by 24.1%--96.1% while retaining 64.9%--98.5% of LLM throughput. All measured LLM outputs arrive within their task-defined freshness windows. On OnePlus 13, for example, camera deadline misses fall by 91.3% while 90.4% of LLM throughput is retained.

0.26

Ripple: Deadline-Aware CPU-GPU Sharing for Embodied Inference

cs.DC cs.AIcs.OScs.PF ·2026-10-31 ·Jiadong Zhang, Xiaosong Ma
similarity 0.62
Abstract

Unified-memory edge devices increasingly run LLM/VLM inference alongside time-critical CPU tasks for perception and control. GPU memory traffic can overwhelm these tasks, yet platforms such as DGX Spark expose neither bandwidth counters nor direct bandwidth regulation. We present Ripple, a CPU-GPU co-scheduler that combines model-derived GPU memory volumes with CPU completion feedback to regulate interference. Ripple divides inference into pausable, layer-based units and translates each CPU job's deadline slack into a GPU-byte budget, admitting work within the budgets of all overlapping jobs. Online updates track changing interference, while deadline-aware pacing and judicious SM footprint narrowing preserve inference progress. Implemented with SGLang engine hooks and a central scheduler, we evaluate Ripple on DGX Spark with four robotic CPU task mixes and LLM/VLM inference. It is the only evaluated approach that consistently meets the 2% CPU deadline-miss target and VLM deadlines across the tested deadline settings while retaining best-effort LLM throughput.

0.17

Dissecting How Die Scaling Breaks GPU Fine-grained Scheduling

cs.AR cs.DC ·2026-09-21 ·Xiaoze Fan, Jianhao Wang, Weihao Cui, Han Zhao, Zhuobin Huang +6 more
similarity 0.62
Abstract

Modern GPUs are no longer physically symmetric. Die scaling leads to both manufacturing-driven floorsweeping and cache and memory partitioning. The former creates chip-specific compute topologies, while the latter causes non-uniform memory access. These asymmetries are substantial. Topology-oblivious compute unit allocation can lead to up to 1.33x performance variation, while remote accesses increase HBM latency by up to 67% and nearly double L2 latency. However, these asymmetries are hidden behind the GPU's logical resource abstractions and can vary across chips. We develop lightweight characterization methods to uncover per-chip compute topology and memory affinity. We then use the discovered information to make existing fine-grained scheduling asymmetry-aware, considering not only how many resources are allocated but also which physical resources are assigned. Across full-GPU kernel execution, intra-application multiplexing, and inter-application co-location, asymmetry-aware scheduling improves mainstream kernels by up to 1.22x, multiplexed LLM inference by up to 14.3%, and avoids up to 1.33x performance variation.

0.38

A principled approach for energy-efficient training via phase-aware GPU frequency tuning

cs.DC cs.LG ·2026-09-21 ·Miguel Braga, Júlio Pinto, Rahma Nouaji, Olivier Michaud, Bettina Kemme +3 more
similarity 0.62
Abstract

Modern AI model training imposes unprecedented computational demands, making it a key contributor to datacenter energy consumption. Yet a significant fraction of the energy consumed during training does not translate to useful computation due to bottlenecks throughout the training pipeline. We present PAFT, a phase-aware, dynamically adaptable GPU frequency tuning system that reduces energy consumption of training workloads with minimal performance overhead. The key insight behind PAFT is that bottlenecks represent an energy optimization opportunity, rather than purely a performance problem: when GPUs are bound to stall, PAFT opportunistically reduces their clock frequencies to match the pace of bottlenecked devices, saving energy without impacting execution time. PAFT achieves this by continuously monitoring pipeline behavior and applying fine-grained frequency adjustments, adapting to workload and system changes. Experiments conducted on twelve widely used models show that PAFT consistently outperforms all baselines, achieving energy savings of up to 46% with an average overhead of 4%.

0.14

Towards Easier-to-Tune Training Algorithms: New Results from AlgoPerf

cs.LG ·2026-10-09 ·Priya Kasimbeg, Ahmed Khaled, Aahlad Chandrabhatta, Yifan Wang, Andy Liu +7 more
benchmark similarity 0.62
Abstract

Evaluating deep learning optimizer is challenging because confounding factors such as hyperparameter tuning and optimizer-model co-design often obscure true algorithmic performance. We present results of the updated AlgoPerf V1 benchmark, in which we evaluate a new set of training algorithms on wall-clock time-to-target across nine workloads on fixed hardware. This report focuses on the results of the benchmark's self-tuning ruleset, which requires submissions to use a single configuration across all workloads. Out of the 13 submissions in PyTorch and JAX, Schedule-Free AdamW ranked first. In addition to the wall-clock scores, we also analyze the trade-offs between step efficiency and wall-clock execution, revealing that matrix-preconditioned and two-loop methods achieve good sample efficiency but require more aggressive computational optimizations to be competitive in runtime. Finally, our hyperparameter saliency and retention analyses show that the learning rate and momentum are highly sensitive across domains and algorithms, highlighting the need for methods that are more robust against hyperparameter tuning and can generalize well to new tasks.

0.11

SLICE-CiM: Selective Layer-wise Inference Control Engine with Confidence-Based Early Exit for SRAM based Digital Compute-in-Memory

cs.AR cs.NE ·2026-10-27 ·Pratham Sharma, Mohd Faisal Khan, Santosh Kumar Vishvakarma
similarity 0.62
Abstract

Digital SRAM compute-in-memory (CIM) macros enable energy-efficient CNN inference by eliminating off-chip weight transfers; however, peripheral control logic introduces three inefficiencies: bit- slice over-activation at reduced precision (L1), unnecessary accumula- tion depth (L2), and idle precision-transition stalls (L3). To address these, we present four synthesizable peripheral blocks: a Flush-Merge Enable (FME) controller, a Confidence-Based Early-Exit (EE) module, a Precision Register with Decoder (PR/PD), and a Bit-Serial Cycle Controller (CC), co-designed to minimize switching activity, wordline overhead, and leakage. Implemented in TSMC 65-nm CMOS at 335 MHz, the peripheral framework occupies 1769.760 μm2 post-synthesis and 1982.131 μm2 post-PnR. Gate-level VCD analysis reports 6.0706 μW leakage and 288.2902 μW (post-synthesis) / 322.8850 μW (post-PnR) dynamic power, totaling 328.9556 μW (0.2402% of system power). The EE module dominates dynamic power (58.2%), while WLG dominates area (58.1%). System evaluation on MobileNetV2 (CIFAR-100) shows efficiency scaling from 31.35 TOPS/W (8b/8b, 1.2 V) to 454.91 TOPS/W (2b/8b, 0.8 V), achieving 13.33x improvement without bitcell modification or retraining. The framework targets CNN workloads exploiting sparsity, with Transformer extension as future work.

0.34