CS Paper Radar

Understanding the Power Consumption of Modern High Bandwidth Memory: Experimental Analysis and Modeling Using Real HBM2 DRAM Chips

Ataberk Olgun, Spiros Galanopoulos, Andreas Kosmas Kakolyris, İsmail Emir Yüksel, Haocong Luo, F. Nisa Bostancı, Onur Mutlu

cs.AR cs.PF ·submitted 2026-10-09 19:03 UTC · v1
top venuecode/data

Abstract

We perform the first comprehensive experimental characterization of power consumed by real HBM2 DRAM chips. Our extensive characterization yields 1) the empirical basis for developing new, accurate HBM power models or validating existing power models and 2) key insights into structural and data-pattern-dependent variation of power in HBM2 chips. Out of our 13 new findings and 6 key takeaway messages, we highlight: 1) a read operation consumes significantly higher current than any other operation and 2) the toggled data bus signals in a data transfer burst significantly affect current draw where toggling all HBM2-internal bank group bus signals causes 29.4% higher power consumption than keeping them constant. Leveraging our empirical insights into HBM2 power consumption, we engineer an access pattern that yields 23.6% higher power consumption on average across 36 tested HBM2 chips than the highest-power-consuming JEDEC-specified current measurement loop. Our results have important implications for HBM-based system design and validation. Relying solely on JEDEC- or vendor-specified test loops to bound HBM power could cause system failures and/or thermal throttling in real-world high-power AI/ML workloads. To improve 3D-stacked DRAM power modeling in open source architectural simulation infrastructures, we develop a new HBM power model. Our model predicts the power of 36 real HBM2 chips across diverse workloads (including a real LLM decode workload) with 7.6 -- 8.1% mean absolute percentage error, substantially more accurately than state-of-the-art HBM power models. The power consumption predicted by our model for a power-intensive microbenchmark closely matches (with 2.4% to 5.3% error across HBM3E's specified supply voltage range) the average HBM3E memory power consumed by three tested NVIDIA H200 GPUs. We publicly release our power model at https://github.com/CMU-SAFARI/HBM-Power.

Comment
Extended version of the paper titled "Understanding the Power Consumption of Modern High Bandwidth Memory: Experimental Analysis and Modeling Using Real HBM2 DRAM Chips'' to appear in MICRO 2026

Closest papers

Nearest neighbours by meaning across the whole index.

HOMARD: Hammering Off-chip Memory via Aggregate power Rail Disclosure

cs.CR ·2026-10-08 ·Eliott Quéré, Li-Chung Chiang, Carina Fiedler, Thomas Rokicki, Maria Méndez Real +4 more
similarity 0.62
Abstract

Software-based microarchitectural attacks have mainly exploited timing differences to infer information from DRAM. However, defenses and optimizations can prevent timing-based approaches from resolving precise microarchitectural behaviors. We show that unprivileged millisecond- and milliampere-resolution power sensors on modern Arm SoCs can recover DRAM mapping functions and reveal the data dependence of memory accesses without timers. We present HOMARD, a methodology that reads DRAM activity from this telemetry, and two attack primitives. HOMARD-RE is a hierarchy-aware linear-algebraic method that reverse-engineers DRAM addressing functions from power alone. HOMARD-DATA recovers the Hamming weight of a fetched cache line through joint-power rail templating and secret byte values through correlation power analysis on the same rails. Across an AMD ZCU102, a Raspberry Pi 5, and a multi-chip-based LPDDR4 NVIDIA Jetson TX2, HOMARD-RE succeeds where row-buffer timing methods fail. On the Pi 5, HOMARD-DATA classifies 9 Hamming-weight levels at 81.5% accuracy and recovers secret bytes in minutes by correlating rail current with data-dependent bit toggling between neighbors in a row. Both primitives work with an architectural load or a mispredicted Spectre-RSB path on which no value is architecturally read. These results establish unprivileged power telemetry as an independent side channel for profiled microarchitectural attacks, particularly when timing falls silent.

0.29

Toki: Profiling HBM Performance on FPGA Systems with RISC-V Soft Cores and PCIe Host DMA Traffic

cs.AR ·2026-09-22 ·Andrea Galimberti, Andrea Motta, Gianni Antichi, Davide Zoni
publishedcode/data similarity 0.62
Abstract · Presented at the 63rd ACM/IEEE Design Automation Conference (DAC '26), July 26-29, 2026, Long Beach, CA, USA; 7 pages, 7 figures, 2 tables

Programmable RISC-V soft cores are becoming more widespread in data-center scenarios, making it crucial to design efficient systems that deploy them on FPGA chips with HBM memory. Toki, released as open source, is the first hardware-software framework that enables profiling the performance of HBM on FPGA accelerator cards by jointly considering (i) the execution of workloads on RISC-V soft cores instantiated on the FPGA and (ii) the injection of memory traffic from the host system via DMA over PCIe, providing insights that cannot be obtained with synthetic traffic generators alone. Extensive experiments target an AMD Alveo U55C card, deploying up to 60 RISC-V compute cores and stressing its HBM2 memory through real-world applications and microbenchmarks with user-defined access patterns. Results showcase how Toki can effectively profile the impact on HBM performance of the compute cores' organization, the workload's memory access patterns, data locality, contention over the memory controllers, and host traffic.

0.36

MEMPOWER: Efficient Power Management with Fine-grained Memory Analysis and Modeling for HPC Workloads

cs.DC ·2026-08-21 ·Nanda Velugoti, Joseph Manzano, Andres Marquez, Nathan Tallent, Kyle Hale
published similarity 0.60
Abstract

Managing the energy consumption and power efficiency of parallel applications is a significant issue in both HPC environments and in the cloud. As emerging applications continue to push against the memory wall of modern machines, the growing imbalance between compute and data movement creates new opportunities to intelligently tune CPU power consumption. Unfortunately, existing frequency and voltage scaling techniques do not adequately capture fine-grained changes in memory access behavior, rendering the compute/data access imbalance invisible to the components of the system that could capitalize on it, thus leaving potential power savings on the table. In this paper, we propose MEMPOWER, a flexible, model-based approach to exposing compute/data movement imbalance that characterizes the fine-grained memory behavior of parallel workloads. This characterization then informs our automated software framework which can statically instrument the application binary with model-determined voltage/frequency transitions that balance fine-grained changes in memory access behavior with the costs of hardware transitions. Using MEMPOWER, we demonstrate a reduction in EDP of 6% to 42% on a range of HPC benchmarks with minimal impact on execution time when compared to the standard OS/hardware-managed power control mechanism.

0.33

Beyond Host-Side Control: HBM Controller Partitioning Across Host and HBM Base Dies

cs.AR ·2026-11-07 ·Tharunya Yogananda, Wantong Li
published similarity 0.59
Abstract · Accepted to IEEE International 3D Systems Integration Conference (3DIC)

Recent generations of high bandwidth memory (HBM), such as HBM4, allow the base die of the memory stack to be manufactured using a logic process that can accommodate more complex controller functions next to the DRAM. We present the first systematic quantitative study of function-level controller placement between the host and the HBM4 base die on a silicon interposer. We divide the controller into its main functions and evaluate four placements, from a complete host controller to a complete base die controller, with intermediate placements that move maintenance or scheduling to the base die. We extend Ramulator 2 with HBM4 timing and maintenance, model a die-to-die (D2D) link with timing checks at the DRAM interface, and compose area, energy, and temperature from synthesized controller blocks, modeled storage, and interface data. A provisioned D2D link preserves per-stack bandwidth and adds about twice its one-way delay to unloaded latency. Maintenance moves to the base die with no bandwidth cost and frees host command slots, and the freed share grows with stack height. Replacing the wide interface with a D2D link saves 61% of host shoreline and 16.85 mm2 of host area per stack. The D2D link dominates the added base die area and power, and the base die runs cooler under streaming with link energy below 0.4 pJ/bit. Shared control delivers up to 1.72x the throughput of static partitioning between two requestors.

0.36

A Unified Device-Level Physical Framework for DRAM Read Disturbance

cs.AR cond-mat.mes-hallphysics.app-ph ·2026-11-13 ·Zhaoqiang Bai, Haocong Luo, Xinhe Wang, Xujing Li, Yuelin Jiang +8 more
similarity 0.58
Abstract

Read disturbance in dynamic random-access memory (DRAM) depends on both local device interactions and the electrical conditions imposed by shared array circuitry. Here, we combine DIMM-level characterization results, device measurements, and TCAD simulations to interpret RowHammer, RowPress, and ColumnDisturb within a unified device-level physical framework. We relate charge evolution at the floating storage node to three key mechanisms: trap-assisted electron migration (EM), capacitive coupling (CC), and sense-amplifier-mediated disturb (SAMD). EM is local and trap-sensitive, while CC weakens with wordline separation. SAMD extends disturbance through shared column circuitry by modifying unselected-cell leakage under rail-level bitline biases, a contribution supported by the approximately ninefold higher OFF-state leakage measured relative to retention-like conditions for a high-voltage storage state. These distinct spatial ranges account for localized RowHammer and RowPress failures and remote ColumnDisturb populations with abrupt changes at subarray boundaries. We further interpret prolonged RowPress activation as a transition from an initial RowHammer-like transient to sustained leakage driven mainly by SAMD under held wordline and bitline biases. Within this device-level framework, trap effectiveness and storage-state-dependent leakage provide an interpretation of previously puzzling failure signatures, including the earlier onset of 0-to-1 RowHammer failures despite the predominance of 1-to-0 failures at high hammer counts. The same framework explains the observed RowPress and ColumnDisturb polarities and the duration-dependent reversal of neighboring- and passing-wordline effectiveness. These results provide a physical basis for developing compact DRAM reliability models across access patterns and array configurations.

0.25

Mitigating Row Buffer Contention Through Multiple Row Buffer (MRB) DRAM

cs.AR ·2026-11-19 ·K Chitra, Arjun Dey, Aryabartta Sahu, Minesh Patel
similarity 0.58
Abstract

DRAM chips contain multiple banks to handle sev- eral memory requests in parallel. However, requests to different rows of the same bank (e.g., from different cores or workload access patterns) interfere with each other by contending for a single shared row buffer. This introduces row buffer conflicts, which incur performance and energy overheads. In this work, we reduce row buffer conflicts caused by interference by introducing multiple row buffers in each bank. We propose multiple row buffer (MRB) DRAM, which employs a primary global row buffer along with multiple secondary SRAM row buffers in the DRAM chip. These secondary SRAM row buffers act as a cache that enables each bank to serve requests to multiple rows in parallel, thereby improving performance and reducing row buffer conflicts. Our evaluation shows that MRB-DRAM with four SRBs per bank improves performance by an average of 29.7%, 24% in singlecore, multicore systems, respectively, compared to a conventional DDR4 baseline with a modest area overhead of 2%.

0.22

SBA: A Practical Memory Performance Denial-of-Service Attack

cs.AR ·2026-11-12 ·Amin Mamandipoor, Connor Rudy Sullivan, Cole Ridge Strickler, Mohammad Alian, Heechul Yun
top venue similarity 0.57
Abstract · Accepted for publication at the 3rd Microarchitecture Security Conference (uASC 2027)

Modern cloud platforms use memory bandwidth regulation to isolate co-located workloads, implicitly assuming that high aggregate bandwidth consumption implies high interference. We show that this assumption is fundamentally flawed. We present the Single-Bank Attack (SBA), a practical memory denial-of-service attack that induces extreme cross-core slowdowns---over three orders of magnitude on memory-intensive victims (e.g., LLM decode)---while consuming less than 1 GB/s of memory bandwidth. SBA exploits the gap between peak and worst-case DRAM throughput by concentrating memory write operations on a single bank, forcing persistent row conflicts and frequent write draining, thereby collapsing the memory service rate. Across Intel and AMD systems, including DDR5- and HBM2E-based platforms, SBA consistently causes orders-of-magnitude slowdowns under realistic multi-tenant settings, revealing a fundamental limitation of bandwidth-centric memory isolation. We further show that this localized contention propagates system-wide: single-bank congestion saturates memory controller queues and induces back-pressure into shared cache and coherence structures, ultimately throttling memory request injection across cores. Consequently, severe interference can arise even at low aggregate bandwidth, evading existing bandwidth-based QoS mechanisms such as Intel RDT MBA. To address this limitation, we propose IQ-MemGuard, a lightweight, software-based memory bandwidth regulator that detects sustained contention using microarchitectural signals and selectively throttles pathological request injection. Our implementation on an Intel platform effectively mitigates SBA, whereas Intel's hardware QoS mechanism fails completely.

0.46

HBFSim: A Simulator for Exploring HBF in Memory Hierarchies for LLMs

cs.AR cs.ET ·2026-10-16 ·Jinheng Li, Guosheng Wang, Hongchao Du, Shangyu Wu, Qiao Li +1 more
publishedcode/datahighly cited similarity 0.56
Abstract

High Bandwidth Flash (HBF) has recently emerged as a promising technology that provides terabyte-scale accelerator-local capacity at bandwidths approaching HBM, with a potential cost-per-bit advantage over HBM. However, its NAND foundation brings long latency, out-of-place updates, and finite write endurance directly into the accelerator memory hierarchy, making its effects on performance and lifetime difficult to predict. To study this emerging design space, we present HBFSim, an LLM-aware event-driven simulator for heterogeneous HBM+HBF memory systems that preserves application-level memory semantics while modeling flash-internal behavior. HBFSim models HBM, HBF with host-managed flash translation, and conventional capacity tiers in one simulation driven by a calibrated serving frontend. Using HBFSim, we study decoding of Llama-3.1-70B and Qwen3-235B-A22B under three workloads that differ in their input and output lengths. HBF serves data that does not fit in HBM 11.0--19.0$\times$ faster than host DRAM; HBM is better used to hold the running sequences' KV cache than as a cache for HBF; and at a published peak load of Llama-3.1-70B, HBF lasts 2.0 to 6.9 years, in proportion to the stacks that share the KV writes. The artifact is available at https://github.com/FlashLabAE/HBFSim and will track public HBF device data as it becomes available.

0.52