CS Paper Radar

PRIM: Energy- and Endurance-Aware Compiler Optimization for Digital Compute in Non-volatile Memory

Patrick Miller, Ulya R. Karpuzcu

cs.ET ·submitted 2026-10-12 17:43 UTC · v1

Abstract

Compute-in-Memory (CiM) is an alternative computing paradigm that circumvents the memory bandwidth bottleneck. Variants utilizing non-volatile memory cells (NVCiM) to perform Boolean operations directly within memory arrays offer significant density and energy-efficiency advantages. Fully realizing this potential, however, requires NVCiM-specific compiler optimizations. In conventional digital NVCiM systems, an output cell must be written -- or preset -- to an operation-specific value before any Boolean operation can occur. Preset-induced extra writes can degrade energy efficiency and cell endurance, often to the point of negating CiM's efficiency gains. In this paper, we introduce PRIM (Preset Reduction In Memory), a compiler optimization framework tailored for digital NVCiM that effectively minimizes preset overhead without compromising computational accuracy or architectural complexity. Unlike existing optimization approaches, PRIM is not restricted to specific types of Boolean operations. Experimental results show that PRIM can reduce execution time and energy consumption by up to 50/% and 44/%, respectively, while boosting cell endurance by up to 50/%.

Closest papers

Nearest neighbours by meaning across the whole index.

Decoupling Disaggregated Memory Optimizations from Indexing: A Compiler-Runtime Approach

cs.DB ·2026-09-02 ·Xinpeng Zhao, Zeling Long, Chaichon Wongkham, Srijan Srivastava, Jiayi Liu +3 more
similarity 0.66
Abstract

Disaggregated memory (DM) decouples compute and memory into independently scalable pools, connected over a slower interconnect rather than a local bus. This decoupling is exactly what makes DM attractive--but it also means that every index must now reason explicitly about remote-memory access and its associated optimizations. State-of-the-art index designs respond to this by embedding remote-memory logic and optimizations directly into their core data structures and concurrency control mechanisms. Consequently, an optimization tuned for one index cannot be lifted and reused in another, and even the same index cannot be ported to a different DM architecture without a fresh round of redesign. This escalating, per-index, per-platform engineering burden is unsustainable as hardware and index requirements evolve. In this paper, we present Nox, a compiler-runtime framework that breaks this coupling by taking an unmodified, concurrent index as input and automatically generating its disaggregated-memory counterpart, without touching the original index logic. A compiler layer rewrites the index's LLVM IR to expose allocation, address, and pointer-dependency information that a centralized runtime uses to drive caching and address translation. Empirically, Nox-generated B+-trees, hash tables, and skip lists scale robustly on real RDMA and CXL hardware across every workload tested. They can also outperform some specialized, hand-crafted indexes and match others, especially on workloads that are closer to real-world ones. These results show that today's fastest disaggregated-memory optimizations need not stay locked inside monolithic, hand-crafted code--a compiler-runtime stack can generalize them while preserving the scalability of proven index implementations, without sacrificing it for portability.

0.47

Heuristic-Driven Resource Optimization for Decoupled Memory Channels on FPGAs

cs.AR cs.DC ·2026-10-18 ·Arnab A. Purkayastha, Shobhit Aggarwal, Jay Tharwani
published similarity 0.65
Abstract · 6 pages. Accepted version of a paper published in the 2026 IEEE International Conference on Interdisciplinary Approaches in Technology and M…

Determining OpenCL channel depths for Decoupled Access-Execute (DAE) architectures in High-Level Synthesis (HLS) is critical for balancing performance and hardware efficiency. Conventional static analysis tools often over-provision these memory channels to ensure performance safety, leading to significant resource stranding. This paper presents an expansion of an LLVM-based static analysis framework, originally proposed for automated DAE generation, by introducing a hardware-aware refinement layer. We identify a counter-intuitive Performance Inversion Paradox linked to baseline memory stalls, where high-stall, grid-based kernels reach peak performance at minimal channel depths and suffer up to a 61% throughput degradation when depths are increased. We propose a Heuristic Refinement Layer that optimizes OpenCL channel depths based on this correlation between baseline stalls and performance saturation. Experimental results demonstrate that our expanded framework successfully identifies the Minimal Peak Performance point across various benchmarks. By avoiding the performance traps of standard over-provisioning, we achieve a 2.52x improvement in Resource Efficiency for an illustrative HotSpot application. This approach maintains peak execution speedup while ensuring the leanest possible hardware profile, outperforming conservative tool-based static predictions.

0.40

Compiling Specialized GPU-Initiated Networking

cs.DC cs.LGcs.PL ·2026-11-13 ·Javid Baydamirli, Ismayil Ismayilov, Kaan Oktay, Didem Unat
code/data similarity 0.64
Abstract · 20 pages, 13 figures, 6 tables. Equal contribution from first two authors. Code: https://github.com/ParCoreLab/aboleth

Machine learning systems specialize computation to the workload and hardware, but communication remains in generic libraries whose device code and transport decisions are not visible to high-level compilers. We present Obaleth, a compiler that makes GPU-initiated communication part of a kernel's intermediate representation, co-optimizes it alongside computation, and lowers it to specialized transport code. Obaleth plans completions around buffer lifetimes and optimizes memory ordering, synchronization, and batching across NVIDIA and AMD GPUs. On H100 GPUs, Obaleth sends small messages up to 3.3$\times$ faster, and on 256 GPUs runs all-to-all 1.25$\times$ and 2.3$\times$ faster than NVSHMEM and NCCL GIN, with much smaller memory and register use. In vLLM, it serves Qwen3-235B on 16 B200 GPUs and answers requests up to 1.29$\times$ faster than with DeepEP.

0.39

GPU Offload in Rust: Portable, Safe, and Fast

cs.PL ·2026-08-13 ·Manuel S. Drehwald, Marcelo Domínguez, Kevin Sala, Alán Aspuru-Guzik, Johannes Doerfert
similarity 0.64
Abstract · 13 pages, 5 figures

High-performance GPU programming has traditionally forced a compromise between execution efficiency and memory safety. While Rust guarantees compile-time memory safety for host CPUs via its strict ownership model, applying these constraints to massively parallel GPU execution environments has previously mandated either vendor-locked Domain-Specific Languages (DSLs) or escaping to explicit unsafe raw pointers. This paper presents a zero-overhead, multi-vendor GPU compilation framework built natively into the Rust compiler (rustc) and LLVM backends. We leverage Rust's rich type system, ownership system, and strict aliasing guarantees (noalias) to efficiently manage and optimize data transfers through LLVM's Offload infrastructure. We expose the technical challenges of cross-vendor ABI lowering mismatches between Host and Device targets and introduce a two-pass compilation pipeline capable of safely handling both manual and compiler-generated memory movements. Evaluating our framework on RAJAPerf demonstrates that our rustc-based solution can generate competitive LLVM IR for GPU kernels, achieving a solid kernel performance against native, hand-optimized CUDA and HIP C++ baselines.

0.39

SARIA: Persistence-Aware KV Caching with Coordinated DRAM--SSD Reload for LLM Agents

cs.OS ·2026-10-22 ·Hui Li, Qihui Zhou, Xiao Yan, Shumeng Zhang, Zijie Dai +5 more
similarity 0.63
Abstract · 12 pages, 16 figures

Multi-turn LLM agents usually reuse long context prefixes, making key value (KV) caching crucial for reducing prefill latency. SSDs provide a low-cost extension to limited DRAM capacity but introduce two key challenges. Eagerly persisting all KV cache blocks incurs substantial write traffic and rapidly consumes finite SSD endurance. Meanwhile, the small number of SSDs provisioned in typical GPU servers limits aggregate reload bandwidth, making SSD-resident KV cache much slower to retrieve than that in DRAM and inflating prefill latency. We present \textsc{Saria} (Selective Admission and Reload for Inference Acceleration), a KV caching system that jointly manages DRAM and SSD by exploiting the session-level temporal locality of LLM agents to reduce SSD writes and accelerate cache reload. On the write path, \textsc{Saria} uses eviction and reuse history to selectively persist KV cache blocks that exhibit reuse opportunities beyond their DRAM residency, avoiding unnecessary writes for KV cache already hosted in DRAM. On the read path, \textsc{Saria} promotes selected SSD-hit KV cache blocks from GPU memory into DRAM after prefill, allowing subsequent reuse within the same session to reduce SSD read traffic while limiting DRAM contention. A transfer coordinator paces DRAM reads while SSD transfers are active, allowing both paths to make progress and efficiently share GPU ingress bandwidth. We implement \textsc{Saria} on LMCache MP and evaluate it using workloads from TraceLab and WEKA, covering both full-attention and hybrid-attention models. The results show that \textsc{Saria} reduces SSD writes by 57.1--58.9\% compared with popular full duplicate caching and the mean time to first token (TTFT) by 42.3--79.7\% over DRAM-only caching.

0.08

A Hybrid GPU and Processing-Near-Memory Architecture with Shared DRAM for Efficient Prefill-Decode Disaggregation

cs.AR ·2026-10-19 ·Sanghyeok Han, Inseong Choi, Byungkuk Yoon, Gyeonghwan Park, Jae-Joon Kim
similarity 0.63
Abstract · 15 pages, 16 figures

The growing adoption of large language models (LLMs) has significantly increased the compute and memory demands of inference serving. A key challenge arises from the mismatch between compute-bound prefill and memory-bound decoding: when executed together, the two phases interfere, while separating them via prefill-decode (PD) disaggregation leads to substantial resource underutilization. Specifically, prefill leaves memory bandwidth underutilized, whereas decoding underutilizes compute resources. Prior approaches, such as GPU Streaming Multiprocessor (SM) partitioning, attempt to improve utilization by spatially dividing GPU resources. However, these approaches remain fundamentally limited, as memory-bound decoding still requires a large number of SMs to saturate bandwidth, reducing the compute resources available for prefill and resulting in an inherent trade-off. To address this limitation, we propose DuoPD, a hybrid PD-disaggregation architecture that integrates lightweight Processing-Near-Memory (PNM) units into the HBM logic die. Unlike prior PIM/PNM designs that rely on costly internal bandwidth scaling and offload only partial decoding operations, DuoPD enables full decoding execution on PNM by leveraging bandwidth sharing with the GPU over a shared HBM interface. This allows compute-bound prefill and memory-bound decoding to execute concurrently with minimal interference, improving both compute and memory utilization. To realize this design, we develop a compact PNM microarchitecture, a TSV-level bandwidth sharing mechanism, and a hierarchical data mapping scheme tailored for hybrid GPU-PNM execution. Evaluations on server-scale LLMs demonstrate that DuoPD achieves up to 2.23x higher end-to-end throughput than conventional PD-disaggregation.

0.24

OpenMP Meta-Lowering: A Declarative Approach to Performance Portable Parallel Code Generation

cs.PL cs.DC ·2026-10-02 ·Luca Parigi, Giuseppe Tagliavini
published similarity 0.63
Abstract · 16 pages, 8 figures, 2 tables. Accepted at CGO 2027 (Salt Lake City, UT, USA)

The increasing diversity of parallel hardware challenges existing compilation flows. While OpenMP provides a portable abstraction for shared-memory parallelism, existing compilers tightly couple the frontend semantics with fixed lowering strategies. This design limits performance portability across different runtimes and architectures. In this paper, we present a modular approach to parallel code generation based on the Multi-Level Intermediate Representation (MLIR) framework. Our approach combines an OpenMP frontend with a domain-specific language (DSL) that specifies how OpenMP constructs are lowered for a target. We call this approach OpenMP meta-lowering, treating lowering as a programmable component rather than compiler-specific logic. This design provides explicit control over code outlining, data sharing, and runtime interfacing across diverse targets. We evaluate our approach on PolyBench/C-OMP across general-purpose and embedded multicore targets. Our results show that the proposed method matches the performance of state-of-the-art compiler toolchains with negligible code-size overhead (< 0.7%). Preserving OpenMP constructs until late lowering stages enables both general-purpose and OpenMP-specific MLIR optimizations. By decoupling lowering from compiler internals, our approach expresses the same OpenMP subset in 5,182 lines of code, ~32% fewer than the corresponding lowering code in Clang and ~76% fewer than in GCC, while supporting the pmsis runtime costs only a few specification lines, enabling rapid support for new runtimes without modifying the compiler.

0.38

ZipFusion: Fusing Lossless KV Decompression with Attention for Efficient LLM Decoding

cs.AR cs.DC ·2026-10-24 ·Yuetao Chen, Peng Wang, Hong Xu
similarity 0.63
Abstract

The key-value (KV) cache places growing demands on GPU memory capacity and bandwidth as large language models serve longer contexts and more concurrent requests. Its footprint limits concurrency, while repeated KV reads during attention can make decoding memory-bandwidth bound. Prior lossless KV cache compression methods reduce transfers between devices but restore the uncompressed KV cache in GPU memory before attention. They therefore leave these repeated KV reads unchanged and require additional work and memory for decompression. We present ZipFusion, which enables attention to consume the losslessly compressed KV cache directly. ZipFusion co-designs the compression format and attention kernel to reduce the bytes read from GPU memory during attention. The kernel reconstructs the original values directly in registers using simple bit operations, avoiding a separate decompression pass and easing the memory-bandwidth bound. ZipFusion also reassigns memory freed by compression across layers to store additional KV cache. Its memory manager reserves space for ongoing generation and makes the remaining capacity available to new requests. We implement ZipFusion on top of FlashInfer and integrate it with SGLang. On an NVIDIA H100, ZipFusion achieves about 1.25x speedup over FlashInfer in decode attention. With 8K context, it improves model decode throughput by up to 26.3% over uncompressed SGLang under the same GPU memory budget. At each method's largest supported batch, it also delivers up to 2.76x the throughput of the best-performing lossless KV cache compression baseline.

0.09