CS Paper Radar

RE-CAM: Readout-Enhanced Content-Addressable Memory for Quantitative Similarity Search

Yi-Chun Liao, Dongyoung Lee, Jiazheng Chen, Tergel Molom-Ochir, Tomohisa Kawakami, Tania Roy, Ik-Joon Chang, X. Sharon Hu, Hai "Helen" Li, Yiran Chen

cs.AR cs.ETcs.LG ·submitted 2026-11-19 01:01 UTC · v1

Abstract

Similarity search is essential to many applications but incurs massive data movement when executed on conventional architectures. Content-addressable memory (CAM) is a promising hardware substrate, as it allows similarity search directly in the CAM array, minimizing data movement and offering both low latency and high energy efficiency. However, conventional CAM array readout produces only binary match results, discarding quantitative similarity information that could be further exploited. Extracting this information requires interpreting the CAM array output voltage that evolves during a CAM search, making the sensing timestamp a crucial design variable alongside the reference voltage and decoding parameter. In this work, we propose RE-CAM, a readout enhancement framework for CAM that is agnostic to memory technology and application, enabling CAM array readout to approximate linear and nonlinear functions with one CAM search. By reusing the comparators to sense the time-varying output voltage multiple times, jointly optimizing the three design variables under system constraints and hardware nonidealities, and co-designing the optimization algorithm and reconfigurable hardware, RE-CAM achieves accurate approximation with low energy and area overhead without introducing a readout throughput bottleneck. Across linear, piecewise-linear, sigmoid, and exponential functions, RE-CAM reduces normalized root mean square error by 10.89% to 24.34% for general approximation and 29.43% to 46.73% when exploiting dataset distribution, compared to a shared uniform-reference ADC baseline with the same comparison budget. Relative to the same baseline, RE-CAM also achieves $8\times$ higher search throughput while reducing energy per search and area by 47% and 39%, respectively.

Closest papers

Nearest neighbours by meaning across the whole index.

CAIMS: Scalable Cross-Array In-Memory Search

cs.AR cs.ETcs.LG ·2026-11-18 ·Yi-Chun Liao, Liu Liu, Ruiyang Qin, Yiran Chen, X. Sharon Hu
similarity 0.66
Abstract

Search, especially top-k similarity search, plays a vital role in many applications, but its performance can be severely limited by memory access overhead. In-memory search (IMS), an alternative to CPU/GPU-based approaches, exploits massive parallelism within a content-addressable memory (CAM) array to perform search in situ and provides both low latency and high energy efficiency. However, the capacity of a single CAM array is often insufficient for modern workloads, requiring data to be distributed across multiple arrays. Deploying scalable top-k similarity search across multiple CAM arrays poses a fundamental problem that existing designs fail to address: how to accurately and efficiently reduce local binary match results to global top-k results. This work introduces CAIMS, a cross-array IMS framework that is agnostic to memory technologies and applications, enabling scalable and efficient top-k similarity search while preserving accuracy. To enable accurate top-k result reduction, CAIMS leverages the existing threshold match functionality with a combined offline and online parameter selection approach to transform local binary match results that lack globally comparable similarity information into global coarse-grained similarity scores. To further improve efficiency, CAIMS conducts hierarchical reduction and transfers only the selection results and necessary control signals with lightweight local peripherals. Implemented on a 3D NAND Flash model and demonstrated using the retrieval phase of retrieval-augmented generation, CAIMS improves throughput by up to $22.3\times/10.1\times$ and energy efficiency by $426.6\times/23.7\times$ compared to the server-class CPU baseline and prior 3D NAND Flash accelerators, respectively, while maintaining comparable accuracy.

0.19

Interface-Aware KV Cache Quantization for Dense On-Chip NVM in Long-Context LLM Decoding

cs.AR cs.AI ·2026-09-04 ·Jiahao Zheng, Yifan Qin, Xiaobo Sharon Hu, Yiyu Shi
published similarity 0.59
Abstract · IEEE/ACM International Conference on Computer-Aided Design (ICCAD 2026)

The key-value (KV) cache is the dominant memory bottleneck in long-context large language model (LLM) decoding: every step reads it entirely, so decoding is memory-bandwidth bound. Holding a quantized KV cache in dense on-chip non-volatile memory (NVM) removes the off-chip transfer. Existing KV quantization methods, however, were designed for GPU-style memory systems: KIVI attaches per-group metadata, adding about 25% to the stored KV cache; KVQuant keeps sparse full-precision outliers that a dense array cannot hold in place. This paper examines what these structures cost when the KV cache resides in NVM behind fixed-range converters, and designs a quantization scheme matched to that interface. The architecture stores the quantized KV cache in dense on-chip NVM, uses a small static analog crossbar only for the fixed rotation, and keeps attention in on-chip digital logic. A randomized rotation and per-vector normalization give every coordinate the same range, so one fixed codebook for keys and one for values, each shared across all tokens of the corresponding tensor type, serve the entire KV cache. The codebook thresholds are programmed once as the read converter's reference levels, enabling fixed-range digitization with no per-token converter reconfiguration. Dequantization is a sixteen-entry lookup and one norm multiply; the only per-vector metadata is one scalar, about 3%. Across models from 3B to 14B and contexts to 32k tokens, the four-bit KV cache maintains accuracy under storage and crossbar noise simulated at realistic device levels. KIVI and KVQuant remain more accurate in software; the advantage of our format lies at the memory interface: 3.1-3.6x lower KV read energy than both mapped to the same NVM, and 8x lower metadata overhead than KIVI. The contribution is a KV quantization co-designed with the NVM memory interface rather than a new accuracy record.

0.28

VARA: A Voltage-Aware ReRAM-Based Accelerator for Energy-Efficient Computing

cs.AR ·2026-08-31 ·Peng Dang, Yintao He, Huawei Li
similarity 0.59
Abstract

ReRAM-based in-memory computing (IMC) architectures are widely regarded as a promising approach to alleviating the computational bottleneck of conventional architectures. Since ReRAM crossbars perform matrix-vector multiplication (MVM) in the analog domain, their computational energy consumption is highly dependent on weight and activation distributions. However, most existing ReRAM accelerators focus primarily on weight optimization while paying limited attention to the impact of activations on computational energy consumption, leaving the energy-saving potential of activation sparsity largely underexploited. In this paper, we propose a voltage-aware ReRAM-based accelerator (VARA), along with its accompanying design methodology. Specifically, we first introduce a voltage-aware training (VAT) algorithm that incorporates a preset threshold into the activation function to steer the activation distribution toward zero values, thereby enhancing activation sparsity. Building upon this, we further propose a co-zero activation reordering (CAR) scheme for crossbar-level computation skipping. CAR clusters activation dimensions based on their co-zero correlations and consistently reorders both the activation matrix and its corresponding weights. This process consolidates scattered zero activations into contiguous zero-valued regions to maximize the benefits of crossbar-level computation skipping. Extensive experimental results demonstrate that, with only marginal accuracy loss, VARA reduces the average total system energy consumption by 60.12\% and improves the average system energy efficiency by 2.68$\times$ compared to the baseline, outperforming existing state-of-the-art accelerators for sparse-activation optimization.

0.24

Read What Matters: Query-Adaptive Quantization for KV Caches

cs.LG cs.CLcs.PF ·2026-10-08 ·Siddharth Bhandari, Lucas Gretta, Krishna Balasubramanian, Shiva Kasiviswanathan
theory similarity 0.59
Abstract · 50 pages, 3 figures, 12 tables

KV-cache entries are stored before their future queries are known, but each decoding query needs precision in different places. We study this mismatch using separate budgets for retained bits and bits fetched per query. ReadKV stores each key and value in a progressive code whose prefixes support different reconstruction precisions. For each query, it allocates key-channel prefixes using the query, computes attention from the reconstructed keys, and then allocates value-token prefixes using that attention. Stored entries remain unchanged. Each stage optimizes a calibrated distortion objective under a fixed budget; we prove exact allocation under diminishing refinement gains and relate these objectives to attention-output error. We also exhibit a finite-dimensional attention family where query-dependent access strictly outperforms every query-independent reader at the same read budget, even with unrestricted competing encoders and decoders. Across six base models, reading four bits on average from an eight-bit cache increases C4 perplexity by at most 0.66%, using about one quarter of the logical reads and half the retained capacity of a 16-bit cache. It is consistently more accurate than storing and fully reading four bits at the same payload-read budget. Retaining more bits than each query fetches is aimed at long-context decoding, where the cache bytes moved per step, rather than the weights, dominate cost. Long-context question answering and retrieval on two instruction-tuned models provide additional quality evidence. On the tested 8K-token, batch-one, single-layer workload on an NVIDIA A10G, a restricted eight-bit ReadKV reader with a two-bit mean payload-read budget has 39% lower latency than the tested TurboQuant codec.

0.44

SimSkip: StrongARM-Comparator Similarity Sensing with SAR-Skipping-Aware Training for Analog Compute-in-Memory

cs.AR ·2026-09-21 ·Yufei Liu, Junjie Wang, Shuang Liu, Yang Wang, Yang Liu
similarity 0.59
Abstract · 5 pages, 6 figures, 3 tables. Submitted to the IEEE International Symposium on Circuits and Systems (ISCAS) 2027

Analog-to-digital conversion dominates the energy and area of analog compute-in-memory (CIM) accelerators. This paper presents SimSkip, a successive-approximation-register (SAR) skipping scheme that combines a lightweight StrongARM-comparator similarity detector with SAR-skipping-aware training. When two held voltages are similar, the upper bits of the preceding code are reused to skip SAR comparisons. Detector-induced disturbance extracted from 55-nm CMOS simulations and bit-exact prefix saturation are embedded in the forward pass, and the network learns to tolerate them through quantization-aware training and knowledge distillation. On an ISAAC-based W4A4 WRN-28-10, SimSkip achieves 77.23% CIFAR-100 top-1 accuracy (mean of three seeds) versus 78.4% for the baseline, while reducing SAR comparisons by 48.2% across three hardware-modeled layers. A first-order model estimates a 27.2% net ADC energy reduction after detector overhead.

0.28

AVSG: Accelerated Vectorized Sparse Gather for Efficient KV Cache Offload in Sparse-Attention LLM Serving

eess.SP ·2026-09-29 ·Wenwei Kuang, Xiangyu Wang, Chong Wu, Jun Wang, Weijie Zhang +3 more
code/data similarity 0.58
Abstract · 20 pages, 7 figures, 4 tables

Dynamic sparse attention reduces long-context attention computation by selecting only a subset of tokens, but still requires access to the full KV cache, leaving serving memory-bound. Offloading the KV cache to host memory reduces device memory pressure but places H2D transfers on the decoding critical path. In DSA, substantial overlap in selected KV entries across decoding steps creates an opportunity for device-resident reuse. Exploiting this reuse efficiently, however, presents three critical challenges: costly matching of selected tokens against entries retained in the HBM buffer, uneven distribution of the remaining H2D transfers across accelerator cores, and retaining frequently accessed KV entries within limited HBM capacity. We present AVSG, an operator that addresses these challenges for DSA while preserving its exact token selections. AVSG uses vectorized hash matching to identify reusable HBM-buffer slots and reserve slots for entries requiring transfer, then evenly partitions these H2D transfers across accelerator cores. Lifetime-based buffer management retains frequently accessed KV entries in the device buffer across decoding steps, and shared slot reservations extend reuse across tokens within a multi-token prediction iteration. On a single NPU, vectorized hash matching is 2.80x faster than scalar dual-pointer matching, miss-only transfer raises effective H2D bandwidth by up to 34.44x over request-level assignment, and an 8K-entry buffer reaches a 94.83% hit rate on real requests. These gains translate into end-to-end improvement: on a serving stack processing a real-world production dataset, AVSG reduces time per output token by 39% and increases output throughput by 1.27x relative to the same KV offload layout without HBM-buffer reuse. The code of AVSG is released at https://gitcode.com/cann/cann-recipes-infer.

0.52

ReCIM-HD: A 40 nm On-Device Learning Accelerator with Hyperdimensional Computing via Importance-Based Hybrid Digital-ReRAM CIM

cs.AR cs.ET ·2026-11-18 ·Keming Fan, Chang Eun Song, Soumil Jain, Weihong Xu, Gopabandhu Hota +7 more
similarity 0.58
Abstract · ESSERC'26

ReCIM-HD is an on-device learning (ODL) accelerator that integrates gradient-free hyperdimensional computing (HDC) with resistive RAM (ReRAM)-based analog compute-in-memory (CIM). ReCIM-HD introduces three key techniques: (1) ultra-low-power HDC-based edge ODL without costly back-propagation operations; (2) non-volatile ReRAM-based storage for class hypervectors and CIM similarity search, enabling energy-efficient inference and power-gating during idle periods without data loss; and (3) a hybrid digital-analog CIM inference flow that protects a small accuracy-critical subset digitally to preserve accuracy. Additionally, it employs a Kronecker HD encoder (KE) to reduce encoding overhead and a customized instruction set architecture (ISA) to support flexible programmability. Fabricated in 40 nm, ReCIM-HD achieves 44.17 TOPS/W (hybrid mode) and 75.8 TOPS/W (ReRAM CIM mode), delivering up to 20.1x higher energy efficiency than the state-of-the-art accelerator.

0.19

Efficient and Variation-Tolerant Tree Inference via Compact and Scalable Mapping in Analog CAMs

cs.ET ·2026-11-19 ·Zhicheng Xu, Wenqing Shi, Pengyu Ren, Chao Li, Bo Wen +4 more
similarity 0.57
Abstract · 13 pages, 15 figures

Tree-based machine learning methods are widely adopted for their interpretability and state-of-the-art accuracy on tabular data. These methods face efficiency challenges on conventional architectures due to irregular memory access patterns during tree traversal. Prior work uses analog CAM to accelerate tree traversal, but inefficient mapping requires exponentially more analog CAM columns as tree depth increases. Accuracy is also degraded by device non-idealities such as programming variations and soft decision boundaries. To address these challenges, we propose ACTIVAR, a compact variation-tolerant tree implementation in a dense analog CAM array via software-hardware codesign. On the software side, we introduce an analog-CAM-aware compact tree model that employs hierarchical subtrees with shared features to maximize hardware utilization. This dense mapping introduces redundancy, improving both model accuracy and hardware defect tolerance. Additionally, we develop Variation-Enhanced Training, which utilizes a fuzzy overlap boundary strategy to build models resilient to analog noise. On the hardware side, we design a FeFET-based analog CAM cell with a driver to sharpen decision thresholds and implement a pipelined multi-core architecture to optimize throughput and energy consumption. Across 13 classification and regression datasets, ACTIVAR maintains near-software-level accuracy while reducing hardware overhead by up to 99% compared with prior analog-CAM-based soft-tree designs. Under realistic hardware noise conditions, ACTIVAR limits accuracy degradation to just 3.7% on the MNIST dataset, whereas baseline models suffer a catastrophic 49.6% drop. Furthermore, the ACTIVAR architecture achieves energy efficiency gains ranging from $10^4$ to $10^6$ times over commercial general-purpose processors, establishing it as a highly efficient solution for high-capacity machine learning at the edge.

0.31