CS Paper Radar

AMARA: Compiler-Driven Accelerator Architecture Design for Graphical Probabilistic Sampling

Zhan Chen, Yihan Fu, Wenshuo Yue, Daijing Shi, Chenyun Yin, Mingrui Wu, Xiangyu Zhang, Yun Liang, Bonan Yan

cs.AR cs.PL ·submitted 2026-11-11 08:03 UTC · v1
published

Abstract

Sampling methods are fundamental primitives for training and inference on probabilistic models. However, practical deployment of sampling algorithms such as Markov Chain Monte Carlo (MCMC) remains difficult on general-purpose processors. Existing hardware accelerators improve sampling efficiency, but they are typically specialized for a narrow class of models, graph topologies, or update schedules, making it difficult to support the diverse workloads used in real applications. This paper presents AMARA, an end-to-end compiler-driven accelerator framework for probabilistic sampling on arbitrary probabilistic graphical models. AMARA introduces MCMC-IR, a domain-specific intermediate representation that separates local factorized computation from global update dependencies. Based on this representation, the compiler performs architecture-aware partitioning, scheduling, and memory-access optimization to expose parallelism and reduce irregular graph communication. The optimized program is lowered to a compact ISA and executed on a programmable ASIC accelerator with specialized compute, sampling, and memory support. By co-designing the compiler and the architecture, AMARA bridges the gap between model generality and accelerator efficiency for practical MCMC-based probabilistic computing. We evaluate AMARA across representative workloads with different structures, including grid-structured and irregular energy-based models, and causal inference networks. The results show that AMARA supports a broad spectrum of discrete probabilistic models while achieving high sampling throughput and efficient hardware utilization. AMARA achieves an average of 6.1$\times$ (up to 16.3$\times$) higher sampling throughput than a JAX-GPU baseline on irregular workloads, and offers broader programmability than state-of-the-art accelerators.

Comment
14 pages, 15 figures, HPCA'27

Closest papers

Nearest neighbours by meaning across the whole index.

HELIX: HEterogeneous LLM Inference with FleXible and Fine-Grained Partitioning

cs.SE ·2026-11-04 ·Victoria Clerico, William Simon, Corey Liam Lammie, Orhun Görkem, Athanasios Vasilopoulos +4 more
code/data similarity 0.67
Abstract

LLM inference is inherently heterogeneous, with performance bottlenecks shifting throughout inference phases down to individual operators. This has motivated heterogeneous serving systems that disaggregate execution across specialized accelerators, typically through fixed partitions such as prefill/decode or attention/FFN. However, these approaches commit to a partitioning strategy in advance, limiting their ability to adapt to different workloads and model architectures while forcing emerging accelerators into predefined roles. We present HELIX, a vLLM-based software stack that discovers, compiles and dispatches workload-aware partitions across heterogeneous accelerators at the operator granularity. At engine initialization, HELIX automatically determines hardware-specific operator placements that minimize execution latency, accounting for profiled execution costs, communication overheads, device capabilities, and memory constraints. It combines dynamic programming, piecewise compilation, and heterogeneous subgraph dispatch, enabling accelerators with partial operator coverage to participate in LLM inference without predefined model partitions. HELIX provides an open-source unified framework for exploring, evaluating, and integrating heterogeneous accelerators for LLM inference, lowering the barrier to emerging hardware by exposing them as subgraph executors. In systems combining H100 and B200 GPUs with emerging accelerators, HELIX resolves LLM partitions that multi-query decoding latency by up to 3$\times$ for dense and sparse transformer models and up to 3.5$\times$ for hybrid state-space models, relative to standard attention-FFN disaggregation.

0.31

FenixWalk: Adaptive Sampling for Large-Scale Dynamic Graphs on Heterogeneous Architectures

cs.DC ·2026-11-03 ·Pinhuan Wang, Jianian Fu, Jie Zhang, Shixuan Sun, Qing Wang +3 more
hot direction similarity 0.65
Abstract

Random walks are fundamental to graph analytics, yet executing them efficiently on large, evolving graphs remains challenging. Existing approaches must balance sampling efficiency, update overhead, and memory consumption. Sampling structures optimized for throughput often incur substantial memory overhead and costly maintenance, whereas lightweight alternatives can suffer from high sampling latency on high-degree vertices or skewed weight distributions. These challenges become more pronounced on heterogeneous CPU--GPU platforms, where host memory provides capacity but GPU memory limits the amount of graph data that can benefit from parallel acceleration. We present FenixWalk, a memory-efficient random-walk system that combines unified in-place sampling with selective GPU acceleration for large-scale dynamic graphs. At its core, FenixWalk employs an in-place Fenwick representation that supports both rejection sampling and inverse-transform sampling without duplicating edge-level weight storage. This representation enables expected O(1) weight lookup for rejection sampling, together with O(log d) prefix sampling and updates. To exploit heterogeneous hardware efficiently, FenixWalk maintains the complete graph and its sampling structures in host memory while selectively placing frequently reused vertex-local samplers on the GPU. By transferring compact walker states instead of repeatedly migrating graph partitions, the system reduces cross-device communication and makes effective use of limited GPU memory. Experiments on large real-world graphs and representative random-walk workloads demonstrate that FenixWalk achieves average speedups ranging from 2.06x to 400.57x over existing random-walk systems.

0.27

Verify Before You Fetch: Reliable KV Cache Compression for LLM Inference

cs.AI ·2026-11-08 ·Devanshu Jha, Gaurav Kumar, Agniva Chowdhury, Kumar Krishna Agrawal, Aditya Desai
code/data similarity 0.63
Abstract · Code: https://github.com/epsilon-prime/verifiedKV Lab Page: https://epsilon-prime.github.io

Long-context applications, long-lived agents, and multi-agent systems require large language models (LLMs) to process and retain extensive information. In these settings, the KV cache can exceed GPU high-bandwidth memory (HBM) capacity, necessitating offloading to host memory, where CPU-to-GPU transfers can dominate inference latency. While KV cache compression can reduce data movement, query-agnostic compression that guarantees accurate attention for arbitrary queries is extremely challenging, if not impossible. We introduce VerKV, a hybrid approach that provides $(ε, δ)$ guarantees for approximating attention on any given query. VerKV maintains a compact set of query-agnostic pivotal tokens on the GPU, together with a token-level sketch of the full KV cache, while retaining the complete cache in CPU memory. We introduce a novel stochastic sampling procedure, PoissonComp, to select the pivotal tokens. At inference time, our proposed KVFetch algorithm uses the pivotal tokens and the sketch to estimate the relative error of the resulting attention computation. If the estimate satisfies a user-specified $(ε, δ)$ tolerance, inference proceeds without accessing host memory. Otherwise, VerKV performs query-aware retrieval to fetch additional tokens from the CPU, refining the attention computation to meet the required tolerance. Thus, VerKV enables attention with bounded approximation error while minimizing CPU-to-GPU transfers. We show that PoissonComp outperforms state-of-the-art approaches to KV cache compression. Combined with KVFetch, it achieves near-dense performance (90-100% of dense quality at 32K context length, averaged over RULER and LOFT), using Qwen3-4B, Qwen3.5-9B and Ministral-3-8B, while reducing the GPU memory footprint by 8-32x and keeping CPU-to-GPU memory fetches within 2-23% of the cache per generated token, depending on the model.

0.30

Tiga: Compiling Graph Message Passing at Scale

cs.DC ·2026-09-21 ·Mingyuan Chi
similarity 0.63
Abstract

Graph message passing offers a common way to express learning algorithms, physical simulations, and numerical solvers. Efficient execution depends on interaction structure and data movement, which can be obscured when a program is expressed as a sequence of tensor operations. On memory-constrained systems such as laptops, materializing connectivity and intermediate messages can also exhaust device memory. We present Tiga, a just-in-time compiler that separates the definition of a message-passing program from how its interactions are traversed, computed, and stored. Tiga preserves graph relations and reducer algebra in a multi-level intermediate representation, enabling traversal specialization, fusion of relation generation with aggregation, and reverse-mode automatic differentiation. Backend-specific lowering targets CPUs and GPUs. Its runtime extends execution beyond device-memory capacity by streaming graph partitions from disk through host memory with page-sized device staging buffers; partition ownership and halo exchange extend the same programming model to distributed execution. A Python interface interoperates with ordinary PyTorch tensors and autograd for forward and backward computation. Numerical checks validate outputs and gradients for differentiable workloads. A differentiated geometric workload characterizes the memory--time tradeoff of fused forward and backward execution. Evaluation against matched Torch and PyTorch Geometric baselines demonstrates reduced runtime and device-memory use for generated-relation workloads, while offloaded forward execution processes billion-edge graphs on a single memory-limited GPU. Measurements on heterogeneous devices further characterize the communication and load-balance costs of distributed execution.

0.12

ParaAnya: Accelerating Parallel Diffusion Sampling with Plug-and-Play Output Caching

cs.DC ·2026-09-29 ·Chee-En Yu, Xiao-Xi Tan, Yi-Cheng Lin, Yun-Shao Tsai, Chee-An Yu +1 more
similarity 0.62
Abstract · 5 pages

Diffusion models have achieved remarkable success in generative tasks, but their inherently sequential sampling process introduces a severe computational bottleneck. Recent Parallel-in-Time (PinT) solvers attempt to mitigate this by parallelizing generation across a sliding window of timesteps, advancing the window only when step-wise changes stabilize. However, this overlapping window mechanism forces the network to repeatedly evaluate the same timesteps. When the input variations between iterations are minimal, these redundant evaluations lead to significant computational waste. To address this inefficiency, we propose ParaAnya, an output cache mechanism agnostic to the parallel sampling algorithm that can reduce the number of function evaluations (NFE). ParaAnya caches input-output pairs of diffusion models and reuses the cached output at overlapping timesteps. By dispatching only cache-miss timesteps to GPU workers, our approach eliminates redundant computation while preserving the structure of the underlying algorithms' update rules. We integrate ParaAnya into four representative parallel sampling algorithms and evaluate its performance on Stable Diffusion v1.5. Across four parallel samplers evaluated with DDIM on eight GPUs, ParaAnya provides $1.30$--$2.43\times$ speedups over their uncached counterparts and reduces NFE by up to 70.1\%, reaching up to a $5.62\times$ speedup over single-GPU serial sampling while maintaining comparable CLIP scores.

0.49

Unifying In-Memory Data Analytics through Sparse Compilation

cs.DC ·2026-09-24 ·Anand Jayarajan, Gennady Pekhimenko
benchmarkcode/data similarity 0.61
Abstract

As modern data analytics workloads become increasingly heterogeneous and hardware-intensive, achieving efficient multi-core performance across diverse applications remains an open challenge. We present Reffine, a compiler-based in-memory analytics engine that delivers high performance across a broad range of data analytics workloads. Reffine introduces a novel intermediate representation (IR), grounded in relational algebra and sparse iteration theory, that provides a unified abstraction for data and computation. This representation enables workload-agnostic, end-to-end optimizations such as operator fusion and automatic parallelization across diverse analytics applications. We further develop a sparse compiler backend that translates Reffine IR into hardware-efficient imperative code, achieving high multi-core performance without domain-specific implementations. On the TPC-H benchmark, Reffine outperforms the in-memory analytical database DuckDB by up to $24.9\times$ and the state-of-the-art compilation-based database Umbra by up to $3.2\times$. Reffine also achieves average speedups of $18.3\times$ and $47.9\times$ over Polars and NetworkX on streaming and graph analytics workloads, respectively. Source code: https://github.com/ampersand-projects/reffine

0.26

RPEG: Training Reasoning Language Models Scalably and Cost-Effectively

cs.DC cs.AIcs.CLcs.LG+1 ·2026-11-06 ·Maciej Besta, Leonard Schmidt, Pierre Pang, Patrik Okanovic, Robert Gerstenberger +1 more
similarity 0.61
Abstract

Reinforcement learning (RL) is now central to training Reasoning Language Models (RLMs), but RLM pipelines are difficult to scale and costly to run because each RL step couples multiple large models, heterogeneous stages, cross-step state, and complex asynchronous dependencies. Existing systems typically optimize only parts of this design space. We address this complexity by jointly optimizing the pipeline structure as well as its execution using a novel data-centric graph intermediate representation called RLM Pipeline Execution Graph (RPEG). RPEG exposes structural choices such as bounded asynchrony and actor-critic sharing together with execution optimizations such as stage fusion, fine-grained scheduling, and multi-dimensional model parallelism. It is coupled with a comprehensive analytical performance model for RLM training, jointly capturing compute, communication, memory, and hierarchical rack-, node-, and GPU-level effects. Across systems with up to 256 GH200 GPUs, our distributed runtime engine implementing RPEG improves performance by up to 2x over ReaL and reduces step latency by up to 2.48x over verl, while increasing hardware utilization by up to 30%. Overall, RPEG provides a graph-driven foundation for next-generation training pipelines for frontier reasoning LLMs.

0.12

First hardware demonstration of a continuous-state probabilistic computer with programmable energy-defined sampling

cs.NE cs.ET ·2026-10-22 ·Chao Fang, Hsuan Yao Tu, Yihan He, Xiao Gong, Chin-Fu Nien +1 more
similarity 0.60
Abstract · 22 pages,5 figures

Probabilistic computing has proved useful for combinatorial optimization, but conventional implementations rely on binary stochastic units and sigmoid-based updates. Extending these systems to continuous variables requires binary encoding. Here we introduce a continuous probabilistic computer (CPC) whose continuous probabilistic bits sample bounded states from conditional distributions defined by local energies. Separating random-source generation from conditional sampling provides a route to using physical random sources with different native distributions. A field-programmable gate array (FPGA) prototype updates finite-precision states using digital randomness. The same framework supports product constraints, memory recovery and solution-space exploration. For the tested 7-bit factorization case, fixed-temperature FPGA CPC reduces mean core solving time approximately 33,000-fold relative to the evaluated quadratic unconstrained binary optimization software implementation. The FPGA recovers stored MNIST patterns without pretrained networks, achieving final success rates of 100%. Software CPC reaches 90% reference coverage of the feasible area in approximately 18% of the time required by restarted Adam.

0.31