CS Paper Radar

Beyond Hotness: High-Resolution Memory Characterization of Cloud and LLM Workloads for Tiered Memory with ARM SPE

Jie Ren, Ivy Peng

cs.DC ·submitted 2026-10-23 14:32 UTC · v1

Abstract

Memory demands from data analytics and LLM workloads grow rapidly, driving up memory costs in cloud and data centers. Multi-tier memory has emerged as a promising solution to balance performance and cost. However, effectively leveraging it requires a deep understanding of workload memory behaviors. This work introduces a high-resolution memory characterization methodology for analyzing workload-specific opportunities for multi-tier memory usage. In response to the increasing deployment of ARM-based processors in data centers, we implement our methodology using the recent Statistical Profiling Extension (SPE) for ARM. By characterizing eight memory-intensive workloads from CloudSuite and LLM applications, we identify region-specific access patterns, evaluate the effectiveness of cloud-oriented cache hierarchies, and propose new metrics based on memory idle periods and latency profiles to determine suitable data placement for tiered memory.

Closest papers

Nearest neighbours by meaning across the whole index.

Performance and Cost-Aware Cache Provisioning

cs.PF ·2026-08-10 ·Ridwanul Tanvir, George Kesidis
similarity 0.70
Abstract

While traditional cache policy evaluations fix capacity - often at 0.1% of the dataset - and measure the resulting hit rate, practical edge-cloud deployments require balancing both storage and computational overhead as billed resources. Consequently, system operators frequently focus on a different objective: determining the minimum cache size needed to satisfy a specific Service-Level Objective (SLO) hit-rate. This paper explores this SLO-centric paradigm by analyzing the minimum capacity and execution time each policy requires to hit a defined target. Additionally, we show that dynamically adjusting the segment ratio in segmented policies based on historical workload patterns enhances efficiency. Through evaluations across real-world and synthetic traces, we present a novel hybrid segmented policy that reduces capacity requirements while keeping processing costs low.

0.04

PANEM: A Heuristic Latency Model

cs.AR cs.PF ·2026-08-28 ·Lucas Crowthers, Mahesh Madhav
published similarity 0.69
Abstract · 12 pages, 23 figures, Presented at the 2026 IEEE International Symposium on Workload Characterization (IISWC 2026), Boulder, Colorado

Accurate pre-silicon memory modeling is essential for achieving meaningful representation of workloads on cloud-class many-core processors. Existing options force a poor tradeoff between fidelity and speed: fixed-latency models are fast but misleading, while cycle-accurate DRAM models are costly and difficult to scale across large study spaces or onto single-core environments. This paper presents PANEM, a lightweight event-driven heuristic model that has guided four generations of commercial server-core development at Ampere Computing. PANEM converts bandwidth-latency characterization data into a dynamic request-bytes/latency response, allowing miss latency to adapt to transient demand, queuing pressure, and read/write mix during simulation. Integrated into a single-core flow with configurable system-loading assumptions, PANEM enables realistic bandwidth constraints and contention-aware latency behavior without sacrificing throughput. Across a broad cloud workload trace suite, PANEM avoids the optimistic and pessimistic biases of fixed-latency baselines, yields more reliable conclusions for prefetching and dynamic throttling studies, and materially improves core-resource sizing decisions. These results show that a calibrated, contention-aware abstraction can deliver practical predictive value for industrial design-space exploration at simulation costs similar to fixed-latency models.

0.46

Revisiting Index Performance under High Cache-Miss Penalties

cs.DB ·2026-11-16 ·Mathew M. Kim, Beomseok Nam
similarity 0.69
Abstract

Modern data-intensive applications often query very large datasets, making index performance critical. Recent index structures exploit the fine-grained access capabilities of emerging storage devices such as DCPMM and CXL. However, existing benchmarks typically evaluate indexes in isolation using saturated, closed-loop workloads that emphasize maximum throughput. Such workloads favor highly concurrent designs while overlooking the application work that follows an index lookup and competes for cache, memory bandwidth, and CPU resources. We present AuxBench - a microbenchmark that evaluates indexes as auxiliary access paths within larger application workloads. Using AuxBench, we evaluate four byte-addressable indexes under controlled post-lookup memory activity, concurrency, and request arrival patterns. Our experiments show that post-lookup memory activity can change the relative performance of competing indexes, that concurrency-oriented designs may lose their advantages when substantial application work is introduced, and that time-varying request arrivals can produce severe tail latency near saturation. These findings demonstrate why index-only benchmarks may not adequately predict performance in larger application workloads

0.21

The Storage View Of KV Cache: Characterizing Cache Behavior Across LLM Workloads

cs.OS ·2026-11-06 ·Talia Q. Chen, Daniel Waddington, Narangerelt Batsoyol, Vijay Chidambaram
similarity 0.67
Abstract

Modern LLM serving increasingly extends KV caches beyond GPU memory into host DRAM, SSDs, and remote storage, yet these stores remain largely governed by workload-agnostic policies such as LRU and fixed TTLs. We ask what execution context is needed by storage to make retention, reclamation, and placement decisions across heterogeneous workloads. We characterize KV-cache reuse, lifetime, and footprint at block granularity across 11 production and public traces spanning chat, API serving, RAG, coding agents, tool-using agents, and multi-lineage workflows. We examine reuse sources, recurrence timing and frequency, useful lifetimes, and storage consumption. Across workloads, KV state takes two recurring forms. Single-session state is reused within one execution: its footprint grows as sessions deepen and accumulates across concurrent sessions; it can remain useful through long idle periods but loses value at session completion. Multi-session state recurs across sessions of the same workload, outlives individual sessions, and spans hot prefixes and a large infrequent tail. While prior systems often exploit application-, agent-, or workflow-specific knowledge, our results suggest that three generic signals (a workload identifier, a session identifier, and session completion) expose much of the structure needed for KV-storage management. Trace-driven comparisons show that workload-specific recurrence estimates retain up to 22% more within-session hits and reduce dead state by up to 40%; a session completion signal eliminates the fundamental idle-time reclamation tradeoff for single-session state; and session-aware reuse classification identifies shared state 4-700x more selectively than request-level access counts. These results motivate a small cross-layer interface for workload-aware KV storage without application-specific semantics.

0.14

MEMPOWER: Efficient Power Management with Fine-grained Memory Analysis and Modeling for HPC Workloads

cs.DC ·2026-08-21 ·Nanda Velugoti, Joseph Manzano, Andres Marquez, Nathan Tallent, Kyle Hale
published similarity 0.67
Abstract

Managing the energy consumption and power efficiency of parallel applications is a significant issue in both HPC environments and in the cloud. As emerging applications continue to push against the memory wall of modern machines, the growing imbalance between compute and data movement creates new opportunities to intelligently tune CPU power consumption. Unfortunately, existing frequency and voltage scaling techniques do not adequately capture fine-grained changes in memory access behavior, rendering the compute/data access imbalance invisible to the components of the system that could capitalize on it, thus leaving potential power savings on the table. In this paper, we propose MEMPOWER, a flexible, model-based approach to exposing compute/data movement imbalance that characterizes the fine-grained memory behavior of parallel workloads. This characterization then informs our automated software framework which can statically instrument the application binary with model-determined voltage/frequency transitions that balance fine-grained changes in memory access behavior with the costs of hardware transitions. Using MEMPOWER, we demonstrate a reduction in EDP of 6% to 42% on a range of HPC benchmarks with minimal impact on execution time when compared to the standard OS/hardware-managed power control mechanism.

0.33

Effects of Hybrid CPU and Cache Architectures on Parallel HPC and Cloud Applications

cs.DC ·2026-08-24 ·Nanda Velugoti, Joseph Manzano, Nathan Tallent, Kyle Hale
similarity 0.67
Abstract

Hybrid CPU architectures have entered the mainstream desktop computing with the announcement of Intel's Alderlake architecture. Such a transition to heterogeneous CPU architecture has various performance and power implications on existing parallel workloads. In this paper we study the effects and impact of hybrid core and cache architecture on the performance of highly parallel HPC workloads. We also illustrate interesting thread scaling behavior for parallel workloads and describes the reason for such behavior both qualitatively and quantitatively. We also explore the impact of hybrid cache architecture on parallel shared data HPC applications. Finally, we illustrate that 1) parallel applications with work imbalance (i.e., threads in application perform different amount of work) scale better across hybrid cores when thread affinity is disabled and 2) hybrid cache architecture has very little impact on parallel shared data applications except for some workloads with locks. This work lays the foundation for our future work which focuses on extending this work to model parallel workloads and hybrid CPU architectures to improve their performance in terms of execution time, memory usage and power consumption.

0.07

Rolling Round-Robin Rate: Standard Heterogeneous Throughput for SPEC CPU

cs.PF cs.ARcs.DC ·2026-08-30 ·Mahesh Madhav, Christoph Müllner, Jiangning Liu, Philipp Tomsich, Jeff Baxter
publishedbenchmark similarity 0.67
Abstract · 6 pages, 1 figure, Presented at the 2026 IEEE International Symposium on Workload Characterization (IISWC 2026), Boulder, Colorado

SPEC CPU has long provided a common foundation for comparing processor, compiler, memory-system, and platform performance. Its multi-copy SPECrate mode measures homogeneous throughput by running many copies of the same benchmark at once. That mode remains valuable, but modern cloud and server systems commonly run heterogeneous collections of jobs whose interactions are shaped by shared caches, memory bandwidth, power management, operating-system scheduling, and noisy neighbors. SPEC CPU 2026 introduces Rolling Round-Robin Rate (RRR), an exhibition run style that uses the existing rate suites to generate deterministic heterogeneous multiprogrammed workloads. This paper describes RRR as a benchmark methodology and proposes a SPEC-like scoring model for future RRR reporting: compute per-benchmark average throughput and coefficient of variation from per-copy ratios, compute a SPEC-style geometric mean for each copy across the suite, and average that population of copy geomeans to form a suite score with its own coefficient of variation. RRR therefore preserves the familiar SPEC throughput tradition while exposing richer information about variability, interference, and heterogeneous-system behavior.

0.28

Memory-Access Profiling for Early Multicore Interference Awareness in Embedded Systems

cs.PF ·2026-11-03 ·Jean-Baptiste Lefoul, Ahmad Shahnejat Bushehri, Alaric Chan Lock, Felipe Gohring de Magalhaes, Gabriela Nicolescu
benchmark similarity 0.66
Abstract

Applications executing on different multicore processors may interfere through shared resources. Such interference can introduce execution-time variability and complicate system integration, especially when applications that behave predictably in isolation are co-scheduled. This paper presents a profiling framework for early multicore interference awareness. The proposed approach captures source-level memory-operation traces from C applications with limited source-code modification and combines them with architectural parameters of the target platform. The framework maps memory accesses to cache sets and memory blocks, generates cache-occupancy profiles and heatmaps, and evaluates pairwise interference risks through a progressive three-stage analysis. First, a coarse-grained screening step classifies applications according to their non-stack memory-access behavior. Second, a static multicore cache-profiling step identifies cache sets whose combined footprint exceeds or approaches the hardware associativity. Third, a temporal footprint analysis tracks intervening unique blocks between consecutive accesses to the same memory block and computes Contention Potential Growth to identify execution windows vulnerable to cache eviction. The framework is evaluated using four benchmark pairings executed on a dual-core platform, with hardware performance counters measuring L2 cache misses used as validation references. The results show that the proposed analysis is consistent with the observed high- and low-risk cases and provides conservative warnings for near-capacity configurations. In particular, different applications pairs might produces up to 213.09% increase in L2 cache misses. These results demonstrate that memory-access profiling can provide practical and interpretable early indicators of cache-related multicore interference before exhaustive hardware testing.

0.24