CS Paper Radar

RAPID: Row-Parallel Arithmetic Processing in DRAM

William C. Tegge, João Paulo Cardoso de Lima, Shouzhi Fang, Jeronimo Castrillon, Alex K. Jones

cs.AR ·submitted 2026-10-01 21:24 UTC · v1

Abstract

Processing-using-memory (PUM) architectures perform computation directly within DRAM to reduce costly data movement between memory and processors. Because charge-sharing operations are confined to individual bitlines, existing DRAM-PUM architectures reorganize data into column-oriented, bit-serial representations. This organization is fundamentally incompatible with the row-oriented, word-parallel layouts used by conventional processors and accelerators, requiring expensive data-layout transformations whenever computation transitions between PUM and conventional execution. In this paper, we present RAPID, a Row-parallel Arithmetic Processing-In-DRAM architecture. RAPID augments the DRAM subarray with two lightweight extensions: migration cells that enable localized horizontal data movement between neighboring bitlines and inversion cells that provide efficient in-array logical inversion. These primitives enable RAPID to operate directly on row-parallel, bit-parallel data, preserving CPU-compatible layouts while exploiting the massive parallelism of the DRAM subarray. In particular, RAPID demonstrates that localized horizontal communication is sufficient to realize shallow arithmetic networks and efficient parallel reduction for multiplication, reducing arithmetic latency while preserving throughput and eliminating costly data-layout transformations, all while maintaining the conventional DRAM array organization. We demonstrate the feasibility and overhead of augmenting DRAM subarrays with migration and inversion cells through detailed transistor-level layout and SPICE-validated circuit simulations. Using the RAPID compiler it is possible to evaluate the performance and data reorganization tradeoffs to ensure the best execution across combined CPU and PUM. Evaluating RAPID on 19 MLPerf benchmarks, there is a 5.9x higher end-to-end performance compared to SIMDRAM for DDR4 PUM execution.

Closest papers

Nearest neighbours by meaning across the whole index.

PipeDRAM: A Data-Transposition-Free Processing-Using-DRAM Architecture with Hardware/Software Pipelining

cs.AR ·2026-09-25 ·Geraldo F. Oliveira, Ataberk Olgun, Ismail Emir Yüksel, F. Nisa Bostancı, Pedro H. E. Becker +4 more
publishedcode/data similarity 0.79
Abstract · Extended version of MICRO 2026 paper

Processing-using-DRAM (PUD) architectures exploit the analog operational properties of DRAM to perform bulk bitwise Boolean and arithmetic operations inside memory arrays by organizing data in a vertical layout, where operand bits are stacked along DRAM columns. However, modern computing systems natively employ a horizontal data layout that preserves the cache line abstraction, leverages spatial locality in row buffers, and enables high memory throughput. This fundamental mismatch forces existing PUD architectures to frequently perform data layout transformations between horizontal and vertical formats, incurring significant performance, energy, and system integration overheads. Our goal is to eliminate data transposition overheads in PUD systems at low cost. To this end, we propose PipeDRAM, a PUD architecture that eliminates the need for runtime data layout transformation, enabling PUD operations directly over horizontally laid-out data. PipeDRAM's key ideas are to (i) deterministically reorganize bits inside each memory request to enable a PUD-friendly data placement within a DRAM array in a horizontal data layout, and (ii) employ a pipeline-based execution model that overlaps bit-dependent and bit-independent in-DRAM operations to exploit bit-level parallelism across the memory array. We compare PipeDRAM to different computing platforms. PipeDRAM provides (i) 11.8x, 11.8x, and 80.4x higher performance and (ii) 25.4x, 3.0x, and 38.0x lower energy consumption than three state-of-the-art PUD systems. PipeDRAM incurs low area cost on top of a DRAM chip (1.86%) and CPU die (0.05%). To enable further research on PUD systems, we open-source PipeDRAM at https://github.com/CMU-SAFARI/PipeDRAM.

0.38

Programmable Data Layouts via On-The-Fly Data Transformation

cs.AR ·2026-10-10 ·Cole Strickler, Renato Mancuso, Manos Athanassoulis, Heechul Yun
top venue similarity 0.70
Abstract

The performance of modern computing systems is heavily bottlenecked by the widening gap between processing throughput and main memory latency, a problem exacerbated when an application's logical data access patterns misalign with the physical layout of data in memory. To address this, we propose a hardware-software co-design framework comprising the Data Transformation Language (DTL) and the Data Transformation Unit (DTU). DTL allows programmers to concisely express complex data layout reorganizations, which the compiler translates into efficient hardware routing configurations. The DTU, a hardware engine logically situated between the Last-Level Cache (LLC) and main memory, transparently evaluates these configurations on-the-fly to gather scattered data elements into dense, logically contiguous cache lines. By converting explicit layout transformations into virtual views, this approach eliminates the need for explicit data materialization in software. We evaluate the DTU within a quad-core RISC-V SoC using FireSim, demonstrating substantial improvements in execution speed and memory efficiency.

0.41

COBALT: Column-swapping Optimized Bit-serial Accelerator for LSTM Tasks

eess.SP ·2026-08-19 ·Mohd Tasleem Khan
similarity 0.69
Abstract

Long Short-Term Memory (LSTM) networks continue to be widely deployed for real-time sequential tasks on edge devices, yet their computational demands challenge deployment on resource-constrained hardware. This work introduces COBALT, a bit-serial compressed LSTM accelerator built on a matrix (input)-vector (weight) reformulation of the standard circulant matrix-vector multiplication (MVM) with offset-binary coding. A novel column-swapping scheme operates on partial products at the bit level when generated in pairs, systematically exposing redundancy across output rows to reduce the number of PP generators and selectors. This redundancy is further exploited using a lightweight correction unit that derives a row's output directly from its paired row. Additionally, for block-circulant MVMs, relocating the shift-accumulate and correction units of each sub-MVM further reduces resource usage. The compressed network achieves weight compression of up to 93.6% while maintaining accuracy on the TIMIT and LibriSpeech-100h benchmarks. On a field-programmable gate array, COBALT achieves superior overall efficiency relative to state-of-the-art LSTM accelerators.

0.12

Accelerating LLM Decoding with In-DRAM Parallelism

cs.AR ·2026-11-02 ·Kevin Xu, Tatsuya Kubo, Daichi Tokuda, Shinya Takamaeda-Yamazaki, Issei Sato
similarity 0.66
Abstract

Autoregressive decoding of large language models (LLMs) on edge devices is dominated by memory-bound general matrix--vector multiplication (GeMV) operations. Processing-using-DRAM (PUD) is a promising computing paradigm that can alleviate this bottleneck by reducing processor--memory data transfers and enabling massively parallel computation within DRAM (e.g., across 65,536 columns). However, weight matrices in individual LLM layers (e.g., a QKV matrix in Llama3 8B with 6,144 rows) are typically too small to fully utilize parallelism in PUD. To address this challenge, we propose an algorithm--system co-design for PUD-based LLM inference. At the algorithm level, we enable a subsequent projection to be packed with the current projection by reformulating it as a computation on the current residual state plus a correction for the residual update. Layer redundancy allows this correction to be approximated using fewer bit-serial steps. At the system level, we tailor data mapping and execution to improve utilization of either columns within DRAM subarrays (SIMD) or independently controlled banks (MIMD). Overall, we achieves up to $1.41\times$ end-to-end decoding speedup over the PUD baseline, corresponding to a $2.45\times$ speedup over processor-based execution.

0.26

M3D-FLOW: A Monolithic 3-D-Stackable Processing-Near-DRAM Architecture for Generative Normalizing Flow Applications

cs.AR cs.ET ·2026-10-19 ·Bipin Thapa Magar, Ishan Thakkar, Sudeep Pasricha
published similarity 0.66
Abstract · 13 pages, 12 figures. Presented in CASES 2026 as a part of ESWEEK 2026. Accepted for publication in IEEE TCAD

Normalizing Flows (NFs) have shown strong results in generative modeling, recently reaching state-of-the-art image generation performance. However, their repetitive, sequential operations limit the computational and memory efficiency achievable with GPUs and TPUs. We propose M3D-FLOW, a novel monolithic 3D-stackable (M3D) DRAM-based processing-near-memory (PNM) architecture that eliminates memory bottlenecks using a Cu-Cu hybrid-bonded logic die integrated with an M3D DRAM die. The M3D DRAM provides dense bank integration and a tiered organization with variable access latencies that M3D-FLOW exploits for the efficient processing of different NF workloads. To maximize compute density while minimizing area, energy, and thermal overhead, M3D-FLOW employs hybrid binary--stochastic floating-point multipliers, performing mantissa multiplication in the stochastic domain and exponent operations in the binary domain. A pipelined execution strategy maps sequential NF stages across dynamically partitioned PE groups, and together with the stochastic hardware, enables M3D-FLOW to achieve at least $6.6\times$ higher performance and $3\times$ better energy efficiency than prior PNM accelerators, with only a 2.2% bits-per-dimension (BPD) increase over bfloat16 and 4.1% over FP32. An analysis of stochastic hardware variants further shows that shorter stochastic bitstreams reduce execution time by up to $3.3\times$ at the cost of higher approximation error, providing a concrete design guide for latency--accuracy trade-offs.

0.26

TileDRAM: Compiler-Controlled Tile-to-DRAM Layouts for 3D DRAM GPGPUs

cs.AR ·2026-10-16 ·Guangyuan Xu, Lanxuan Zhang, Ziyang Li, Zhaoshi Li, Yajun Ha
similarity 0.65
Abstract

Modern GPU compilers expose rich layout abstractions for mapping tensors to threads and on-chip resources, but compute-unit-local (CU-local) 3D DRAM architectures lack a corresponding abstraction for specifying how operand-tile data are arranged within the DRAM address space. Without such an abstraction, conventional placement under the existing DRAM address mapping can subject structured tile accesses to severe row-buffer conflicts. We present TileDRAM, a compiler-runtime toolchain that introduces tile-to-DRAM layouts as a compiler-visible abstraction for DRAM placement. TileDRAM extends the GPU Linear Layout abstraction from compute-side mapping targets to DRAM placement coordinates and selects layouts per tile access signature, using recurring tile-local access behavior as the specialization unit. Since row-buffer locality depends on ordered request history, TileDRAM evaluates candidate layouts through their row-buffer behavior under fixed DRAM address mappings. Accesses under the selected layouts are realized through ordinary address arithmetic, without new address-remapping hardware or ISA support. In full-trace replay on Accel-Sim/Ramulator2 across 108 mixture-of-experts (MoE) feed-forward network (FFN) cases, TileDRAM reduces the request-weighted row-conflict rate from 75.7% to 15.2% and achieves a 1.77x geomean speedup. Retargeting six traces to six 3D DRAM organizations yields a 2.20x geomean speedup under the same compiler abstraction. On an RTX 3090, address transformation adds 0.17% geomean overhead over identity.

0.15

Compiling Semi-Ring Dynamic Programming to Tier-Aware 3D-DRAM Processing-in-Memory

cs.AR ·2026-10-06 ·Mahbod Afarin, Tsung-Han Lu, Tajana Rosing
similarity 0.65
Abstract

Processing-in-Memory (PIM) on monolithic 3D (M3D) DRAM is a promising answer to the memory wall for data-intensive dynamic programming (DP), yet extracting its performance today demands hand-written kernels: the programmer must pick a tile size, place data across non-uniform-latency memory tiers, partition work between heterogeneous processing units, and insert the right broadcasts and queues by hand. We present GenMLIR, an MLIR compiler that automates this mapping for tier-aware 3D PIM. GenMLIR encodes the semi-ring generalized grid update, the algebraic form shared by all-pairs shortest path (APSP) and sequence alignment, as a first-class IR abstraction, and lowers it through four GenDRAM-aware pass groups that derive blocked tiling, tier-aware placement, tile-to-PU assignment, and explicit communication. We also characterize precisely which DP recurrences the abstraction admits and which it does not. On the GenDRAM architecture, GenMLIR-generated code runs up to 5.8 (APSP) and 16.7 (alignment) faster than a lowering without compiler support, and 1.2-1.4 faster than a standard affine-tiling PIM compiler we also implement, reaching 90-100% of an achievable-performance bound. It expresses these workloads in 4-9 lines instead of 300-500 and compiles in a negligible fraction of runtime.

0.22

NITRO: High-Performance 3D NAND Flash-Based In-Storage Computing with Enhanced Activation Dataflow

cs.AR ·2026-08-12 ·Sanghun Shin, Sangyeon Kim, Gisan Ji, Sungju Ryu
published similarity 0.65
Abstract · Extended manuscript of DATE 2026

In-storage computing (ISC) is considered a next-generation memory architecture for its ability to relieve the data bottleneck between the host and the memory. While the required resources of large language models (LLMs) have increased significantly in recent years, the memory density has not scaled accordingly. Recently, several works have studied NAND flash-based processing-in-memory (NAND-PIM) schemes to exploit the high density of the memory. However, they do not address the dataflow/buffer for the intermediate values, so a simple method is to deal with the values in the slow flash memory array. To overcome such a limitation, we propose a high-performance NAND flash-based ISC architecture with enhanced activation buffering. Instead of using the very slow flash memory array for the intermediate values, our architecture buffers the values in a fast DRAM subsystem. This approach effectively handles the high-latency penalties when activations are programmed into slower TLC NAND flash. We also introduce a distributed dataflow approach for the NAND-PIM array. This approach maximizes computational parallelism by employing efficient intra-plane data mapping. The results show that our proposed architecture achieves significant performance improvements, reducing the inference latency by up to 85% compared to the baseline.

0.17