CS Paper Radar

Towards Signal Processing In Networks: Multiplierless Block Transforms in Data Plane

Shota Ono, Marika Morikawa, Kazuma Kaneko, Junya Hara, Hiroshi Higashi, Shigenori Tani, Yuichi Tanaka

eess.SP ·submitted 2026-10-09 10:04 UTC · v1

Abstract

In this paper, we propose a signal processing concept, signal processing in networks (SPIN), that executes signal processing tasks directly on programmable switches. Traditionally, signal processing tasks, such as denoising, decomposition, and dimensionality reduction, are performed on edge or cloud servers, i.e., raw streams are transmitted from sensors. This imposes a heavy processing load on edge/cloud servers and requires substantial bandwidth. SPIN addresses this problem by performing signal processing on the forwarding path. As an initial realization of SPIN, we implement a DCT-based block transform on a programmable switch. We redesign its computation using a lifting-based lattice structure, multiplierless approximation, and pipeline partitioning across stages in a programmable switch. We verify at the compiler level that the proposed implementation satisfies the strict hardware constraints of the Intel Tofino2--specifically regarding pipeline stages, packet header vector, and ternary content-addressable memory usage--and therefore enables line-rate processing. We further quantify the per-packet latency of the ingress match-action pipeline. The SPIN-oriented preprocessing adds 46 cycles (approximately 31 ns at 1.5 GHz). The total ingress match-action unit latency is approximately 87 ns, which remains consistent with the low-latency operation targeted by SPIN. Furthermore, we implement the proposed block transform on a BMv2 switch using P4 and evaluate it as preprocessing for in-network machine learning. Experimental results demonstrate that the proposed method improves classification performance over unfiltered baselines.

Comment
Submitted to the 2027 ICASSP-OJSP track, 9 pages, 4 figures

Closest papers

Nearest neighbours by meaning across the whole index.

LLM Inference on IMC-NoC Architecture with Balanced Dataflow and Fine-Grained Parallelism

cs.AR ·2026-09-01 ·Yimin Wang, Yue Jiet Chong, Xuanyao Fong
published similarity 0.58
Abstract · 11 pages, 15 figures, an extended version of LEAP ICCAD'2025

LLM inference has become an essential service, yet it imposes unprecedented demands on memory bandwidth, computational density, and communication efficiency. While IMC is a promising solution to the memory wall issue, the heterogeneous data dynamicity of LLM requires complementary resources to handle intermediate data generated during run-time. Furthermore, the massive number of parameters in LLM necessitates scale-up architectures where on-chip data movement is often the primary performance bottleneck. This article presents a hardware-software co-design framework that unifies distributed compute, memory, and communication into a seamless processing-communication fabric. On the hardware side, we propose a scalable architecture, named LEAP, that integrates IMC PE, NMC PE, and INC. This allows each hardware layer to execute specialized tasks: IMC for static weights, NMC for dynamic data, and INC for partial result reduction. On the software side, we introduce a partitioning, mapping, and scheduling framework optimized for key metrics in LLM serving, including throughput and latency. To address the distinct computational intensities of the prefill and decode phases, we present a prefill-decode disaggregation approach that dynamically reconfigures PE organizations to maximize resource utilization. Compared to commercial GPU platforms, the proposed architecture provides a throughput and an energy efficiency improvement of $\geq{}1.52\times$ and $24.91\times$, respectively.

0.31

Benchmarking StreamNTT with a Verilog-to-Routing Toolchain

cs.AR ·2026-09-19 ·Wei He, Young-kyu Choi, Hyunwoo Park, Sunwoong Kim
code/data similarity 0.58
Abstract · Accepted to the 2nd Workshop on Domain-Specialized FPGAs (WDSFPGA), co-located with ISFPGA 2026

As post-quantum cryptography algorithms move toward large-scale data center deployment, hardware acceleration of their computational bottleneck, which is the number theoretic transform (NTT), has gained increasing attention. StreamNTT, a high-level synthesis- and field-programmable gate array-based accelerator, achieves state-of-the-art throughput through various optimization techniques. However, its reliance on a commercial tool and a device makes direct comparisons difficult for researchers without access. We address this by building StreamNTT on an open-source Verilog-to-Routing toolchain, which achieves similar digital signal processing and multiplier usage. Significant differences in internal memory utilization indicate that further memory-level optimization is needed to approach commercial tool performance.

0.32

Programmable Data Layouts via On-The-Fly Data Transformation

cs.AR ·2026-10-10 ·Cole Strickler, Renato Mancuso, Manos Athanassoulis, Heechul Yun
top venue similarity 0.57
Abstract

The performance of modern computing systems is heavily bottlenecked by the widening gap between processing throughput and main memory latency, a problem exacerbated when an application's logical data access patterns misalign with the physical layout of data in memory. To address this, we propose a hardware-software co-design framework comprising the Data Transformation Language (DTL) and the Data Transformation Unit (DTU). DTL allows programmers to concisely express complex data layout reorganizations, which the compiler translates into efficient hardware routing configurations. The DTU, a hardware engine logically situated between the Last-Level Cache (LLC) and main memory, transparently evaluates these configurations on-the-fly to gather scattered data elements into dense, logically contiguous cache lines. By converting explicit layout transformations into virtual views, this approach eliminates the need for explicit data materialization in software. We evaluate the DTU within a quad-core RISC-V SoC using FireSim, demonstrating substantial improvements in execution speed and memory efficiency.

0.41

FuTransformer: FP8 prefill and INT4 decode for Llama-style inference on Hopper

cs.AI ·2026-10-16 ·Wei Wang
similarity 0.56
Abstract

Transformer prefill reuses weights across tokens and is dominated by matrix multiplication and activation data movement; single-token decode has limited reuse and is dominated by weight and KV-cache reads. We present FUTRANSFORMER, a CUDA implementation of a Llama-style Transformer block for NVIDIA Hopper GPUs. It applies CODA's GEMM-plus-epilogue approach to inference, adding direct KV-cache placement, folding the fixed RMSNorm gain into weights, writing FP8 output directly with delayed scaling, and supporting prefill without padding. Decode uses INT4 projection weights and an INT4 KV-cache copy. Weights and keys follow Tensor Core fragment layouts for direct dequantization into MMA operands without full intermediate tiles. A block-diagonal Hadamard transform reduces cache quantization error from outlier channels. On one H100, prefill is up to 1.48x faster than compiled PyTorch FP8 and reaches 1.29 PFLOP/s. For gate/up projection with SwiGLU at the 8B shape, our BF16 path is 1.13x faster than CODA. Our FP8 path is 2.02-2.18x faster than the evaluated BF16 CODA path. Across 8B and 70B shapes at batches 1-32, INT4 decode is 1.3-2.4x faster than the faster of PyTorch's BF16 and INT4 blocks at each point: 2.1-3.3x faster than compiled, CUDA-graphed BF16 and 1.3-6.3x faster than the same block using PyTorch's INT4 projection kernel, which targets a single query row and loses to BF16 at batch 32. It is also 31-51% faster than our FP8 path. Complete Llama-3.1-8B decode takes 2.8 ms per step at batch one, versus 8.1 ms for the compiled transformers implementation. Decoding every WikiText-2 token through the INT4 weight path raises perplexity from 6.24 to 7.23 with round-to-nearest weights or 6.99 with GPTQ in the same format; the rotated INT4 cache adds 0.20. The kernels match a quantized reference computation within 0.001 perplexity.

0.11

Fast and Memory-Efficient Wavelet Convolutions via I/O-Aware Reformulation

cs.CV cs.AI ·2026-08-11 ·Amit Aflalo, Shahaf E. Finder, Roy Amoyal, Eran Treister, Oren Freifeld
published similarity 0.56
Abstract · Accepted to TMLR

Wavelet convolution (WTConv) has emerged as an increasingly popular drop-in replacement for standard convolutions, expanding a network's receptive field exponentially with the number of decomposition levels while keeping the parameter count linear. However, its reference implementation is severely memory-bound due to excessive data movement through high-bandwidth memory (HBM). We develop an I/O model of WTConv to characterize this bottleneck and use it to guide three algebraic reformulations: (1) recomputing the inexpensive Haar analysis butterfly on chip, (2) collapsing the multi-level synthesis cascade into a single closed-form pass indexed by output-coordinate bits, and (3) folding learned per-channel scales into the convolution weights. Together, these reformulations enable an I/O-aware fused implementation that substantially reduces HBM traffic. We evaluate the WTConvNeXt configuration across decomposition levels and a broad range of tensor shapes. Despite performing comparable arithmetic, the reference WTConv is substantially slower than the depthwise convolution it replaces. Our reformulation reduces modeled HBM traffic by approximately $2.55\times$, yielding up to a $4.35\times$ training speedup over the reference while roughly halving peak memory usage. Thus, our reformulation preserves the benefits of WTConv while substantially reducing its execution time and memory footprint, removing the systems overhead that previously limited its practical efficiency.

0.27

Scheduling Data to Compute: Hierarchical Data Pipelining for LLM Training on AMD GPUs

cs.AR cs.AI ·2026-11-03 ·Gengyu Rao, Ghadeer Ahmed H Alabandi, Su Ann Chong, Thomas Huber, Kailash Gogineni +26 more
code/data similarity 0.56
Abstract

Large language models (LLM) have been the key focus of recent machine learning breakthroughs. As the model size grows exponentially, the computation needed for training is astronomical. A GPU can do a huge amount of arithmetic. However, the data its compute units need is not always available when they are ready. This data may still be in slow memory or on another GPU. While the chip waits, expensive compute sits idle, and at scale even a small drop in utilization translates into substantial cost. In this paper, we treat this waiting time as the primary bottleneck. We focus on hardware features designed for explicit data movement. These features load data from off-chip memory onto the chip, rearrange it into the layout the computation requires as it is loaded, and transfer it between GPUs. By orchestrating them, we keep a steady pipeline of data flowing to the compute units so that they remain busy. We present kernel and framework implementations that address this bottleneck using three key data-movement instructions and hardware features. The Tensor Data Mover (TDM) moves tensor data from high-bandwidth memory (HBM) into on-chip memory. Transpose-load instructions transpose tensor data into the layout that LLM computations require as it is loaded on-chip. Finally, System DMA (SDMA) and Remote DMA (RDMA) move data between GPUs over the scale-up and scale-out networks, respectively. Our results show that TDM and transpose-load provide on average 1.88x and 1.19x speedup at the kernel level, respectively. The SDMA and RDMA lead to on average 1.16x and 1.33x speedup for computation overlapped with communication. This is the first AMD LLM training system to utilize all the GPU data pipelining features. Our implementation has been integrated into open source framework and library for AMD GPUs. https://github.com/AMD-AGI/Primus-Turbo https://github.com/AMD-AGI/Primus

0.34

PipeDRAM: A Data-Transposition-Free Processing-Using-DRAM Architecture with Hardware/Software Pipelining

cs.AR ·2026-09-25 ·Geraldo F. Oliveira, Ataberk Olgun, Ismail Emir Yüksel, F. Nisa Bostancı, Pedro H. E. Becker +4 more
publishedcode/data similarity 0.56
Abstract · Extended version of MICRO 2026 paper

Processing-using-DRAM (PUD) architectures exploit the analog operational properties of DRAM to perform bulk bitwise Boolean and arithmetic operations inside memory arrays by organizing data in a vertical layout, where operand bits are stacked along DRAM columns. However, modern computing systems natively employ a horizontal data layout that preserves the cache line abstraction, leverages spatial locality in row buffers, and enables high memory throughput. This fundamental mismatch forces existing PUD architectures to frequently perform data layout transformations between horizontal and vertical formats, incurring significant performance, energy, and system integration overheads. Our goal is to eliminate data transposition overheads in PUD systems at low cost. To this end, we propose PipeDRAM, a PUD architecture that eliminates the need for runtime data layout transformation, enabling PUD operations directly over horizontally laid-out data. PipeDRAM's key ideas are to (i) deterministically reorganize bits inside each memory request to enable a PUD-friendly data placement within a DRAM array in a horizontal data layout, and (ii) employ a pipeline-based execution model that overlaps bit-dependent and bit-independent in-DRAM operations to exploit bit-level parallelism across the memory array. We compare PipeDRAM to different computing platforms. PipeDRAM provides (i) 11.8x, 11.8x, and 80.4x higher performance and (ii) 25.4x, 3.0x, and 38.0x lower energy consumption than three state-of-the-art PUD systems. PipeDRAM incurs low area cost on top of a DRAM chip (1.86%) and CPU die (0.05%). To enable further research on PUD systems, we open-source PipeDRAM at https://github.com/CMU-SAFARI/PipeDRAM.

0.38

Entwine: Coordinating Tiled Computation and Fine-Grained Communication across GPUs

cs.DC cs.AR ·2026-09-10 ·Kai Ma, Quanfeng Lv, Jingguo Ge, Bowei Dai, Kefan Ruan
code/data similarity 0.56
Abstract · 15 pages, including references and appendices

Modern high-performance GPU computations partition tensors into tiles to exploit data reuse and parallelism. Individual tile computations complete earlier than the full tensor computation, creating opportunities to overlap computation and communication. However, a mismatch between computation and communication progress can limit these opportunities. Communication stalls when no data is ready, and may lag when data arrives in bursts. Communication can also slow computation by consuming shared resources, offsetting the benefits of overlap. We present Entwine, which coordinates tile computation order, fine-grained communication, and SM resource allocation to minimize overall completion time. Entwine reorders tile computation to produce data for communication at a more regular pace. Entwine couples this schedule with fine-grained SM-based communication to process tile results with low latency and low overhead. Since the communication kernel also consumes SM resources, Entwine coordinates their allocation to balance communication progress against computation slowdown. Across representative tensor-parallel LLM workloads, Entwine achieves a geomean speedup of 1.232x (up to 1.433x) over cuBLAS+NCCL, and outperforms state-of-the-art overlap baselines by 3.1-9.8% in geomean. We will open-source our implementation upon publication.

0.53