CS Paper Radar

ConvLUT: Rethinking Convolutional Architectures for Ultra-Low-Latency LUT-Based Inference

Marta Andronic, Olivia Weng, Danial Zuberi, Ryan Kastner, George A. Constantinides

cs.LG ·submitted 2026-11-08 17:21 UTC · v1

Abstract

Lookup-table (LUT)-based neural networks (NNs) offer a compelling path to ultra-low-latency inference on FPGAs. However, existing LUT-based neural networks have largely been restricted to MLP-style topologies, leaving structured vision and audio tasks underexplored. We present ConvLUT, a family of fully unrolled LUT-mappable convolutional architectures targeting the MLPerf Tiny benchmarks. It is comprised of a standard convolutional model built around a hierarchical tree of LUTs and a depthwise separable model following the MobileNet design philosophy. These architectures are enabled by four key contributions: a unified LUT-based convolutional framework supporting standard, depthwise, and pointwise convolutions under explicit fan-in constraints; a LUT-tree that addresses the $k^2c$ receptive field dimensionality bottleneck; a channel-partitioned LUT pooling layer that replaces conventional spatial pooling; and a hardware-aware iterative pruning strategy for stable convergence at scale. Evaluated on CIFAR-10, Visual Wake Words, and Keyword Spotting, ConvLUT achieves competitive accuracy with fully unrolled inference latencies below $\sim 50$ ns. On CIFAR-10 it reaches at least a $8844\times$ latency reduction over non-logic-based prior implementations, and outperforms existing MLP-style LUT-based models on the same task by at least $20$ pp in test accuracy.

Closest papers

Nearest neighbours by meaning across the whole index.

ResoLUT: Distilling Residual Networks into Lookup-Table Networks

cs.LG ·2026-11-17 ·Katarzyna Fojcik, Aidas Mikalauskas, Matthias Kampa, Renaldas Zioma, Jogundas Armaitis
similarity 0.70
Abstract

Logic networks built from Boolean gates or lookup tables (LUTs) promise efficient hardware inference, but training them at depth and learning their connectivity remain difficult. We do not train logic as logic. Instead, ResoLUT distills a trained residual network (ResNet) layer by layer into LUT trees, preserving its multi-bit residual stream and BatchNorm while extracting connectivity from the teacher rather than learning it. To our knowledge, ResoLUT is the first convolutional logic network in computer vision to retain a ResNet-style multi-bit residual stream. Hard LUT decisions are used throughout training and inference, avoiding a discretization gap in the logic. ResoLUT-20 reaches 95.66% on CIFAR-10 with twice the trees, and ResoLUT-56 reaches 78.87% on CIFAR-100 and 53.69% on ImageNet at 64px, the best results of logic and lookup-table networks on CIFAR-10 and CIFAR-100, showing that LUT networks can scale in depth. We explore and quantify the role of the residuals and the tree budget and shape. The most optimized model, a ResoLUT-8 with one eighth of the trees and further optimizations, fits on an FPGA and reaches 90.04% at 189k images per second, the highest throughput among the compared models above 85% on CIFAR-10.

0.13

DiffLUT-Net: Differentiable Training of FPGA LUT Networks with Learnable Connectivity

cs.LG cs.AIcs.AR ·2026-09-08 ·Jiaqi Ye, Xinrui Gong, Jingcun Wang, Olga Kondrateva, Bing Li +1 more
code/data similarity 0.69
Abstract

Field-programmable gate arrays (FPGAs) enable efficient neural-network inference, but most deployment flows either accelerate multiply-accumulate operations or convert pretrained quantized models into lookup tables (LUTs). We present DiffLUT-Net, an FPGA-native network connected by six-input LUTs that are trained from scratch. We jointly learn the 64 truth-table entries of a LUT and the source to each of its six input ports using a differentiable LUT function relaxation and hardware source selection. After training, the truth tables and connections are discretized, unused logic can be pruned, and the network is exported directly as synthesizable Verilog. Across five benchmarks, DiffLUT-Net achieves favorable accuracy-resource trade-offs. These results demonstrate the effectiveness of jointly learning LUT functions and sparse connectivity for compact FPGA-native inference. The code is available at https://github.com/TUDa-HWAI/DiffLUT-Network.

0.31

GaLe: memory-efficient Global Approximate and Local Exact features

cs.CV ·2026-09-02 ·Alberto Ancilotto, Elisabetta Farella
published similarity 0.69
Abstract

Embedded devices typically lack the resources of GPU-equipped machines, and existing inference methods suffer from either high computational overhead (patch-based) or accuracy loss (approximation-based). We propose GaLe, a memory-efficient technique that enables the deployment of pretrained networks on constrained devices without retraining. GaLe partitions feature maps into two components: a local exact (Le) representation that preserves fine details and a global approximate (Ga) representation that retains long-range dependencies. Unlike standard tiling, GaLe supports global operations and attention mechanisms found in hybrid CNN-transformer models. Validated on ImageNet, our method matches exact-inference performance while achieving up to 65% speedup and 90% RAM reduction on a Cortex-M33 compared to patch-based inference. We further demonstrate GaLe's versatility across classification, detection, and generation tasks, highlighting its potential as a foundation for resource-efficient architecture design.

0.30

From Deep to Shallow: Unconstrained and Efficient Layer Merging Strategy

cs.LG ·2026-09-04 ·Petro Shulzhenko, Gabriele Spadaro, Enzo Tartaglione
code/data similarity 0.68
Abstract · 13 pages, 3 figures, accepted at the ITEM Workshop at ECML PKDD 2026

Although Deep Neural Networks have become foundational in many areas of Machine Learning, high computational demands limit their application in resource-constrained environments. To address this issue, depth compression methods have been proposed to identify and linearize redundant activation functions, thereby allowing for the merging of layers without intermediate non-linearities. However, these methods face two key challenges: they cannot be directly applied to convolutions with padding due to the absence of an analytical solution for merging these layers, and they typically increase the kernel size of merged layers, thus limiting speed-up gains. To overcome these limitations, we propose an efficient strategy that enables merging of layers without an existing analytical solution, and also without increasing kernel size. We validate our approach across multiple architectures and datasets, and measure inference speed-up gains on real embedded platforms. We publicly released the code at https://github.com/ShulzhenkoPetr/deep-to-shallow.

0.36

MCL: Meta Convolution Layer

cs.CV ·2026-10-08 ·Naim Reza, Md Al Amin, Ho Yub Jung
similarity 0.67
Abstract

Dynamic convolution enhances convolutional neural networks (CNNs) by adapting kernels to input content, but it expresses the effective kernel as a linear mixture of a small number of basis kernels, which limits expressivity and complicates optimization as the mixture size grows. In this work, we revisit dynamic convolution from a functional perspective and propose the Meta Convolution Layer (MCL), which directly models the convolutional kernel as an input-conditioned function W(x) realized via a high-order polynomial expansion. Leveraging nested residual blocks inspired by deep polynomial networks, MCL implements a structured polynomial meta-network that generates a single input-adaptive kernel, thereby decoupling representational power from the explicit number of mixture kernels and alleviating training instability. MCL is a plug-in addition with standard convolutions and can be seamlessly integrated into both CNN and transformer backbones. Experimental evaluation shows that adding MCL improves the Top-1 accuracy of Resnet- 18, Resnet-50 and ResNet-101 by 6.61%, 3.42% and 3.05% on the ImageNet dataset. Moreover, the proposed method significantly boosts the accuracy of Resnet and Wide-Resnet variants on CIFAR-10 and CIFAR-100 datasets. Additionally, the proposed method outperforms previous methods on fine-grained visual classification tasks using Swin and ViT backbones. These results demonstrate that high-order polynomial kernel generation is a powerful and scalable alternative to linear mixture based dynamic convolution.

0.14

FAME: An FPGA-Based Platform for Approximate Multipliers Evaluation with Pattern-Guided DNN Retraining

cs.AR cs.LGcs.PF ·2026-09-15 ·Rappy Saha, Nima Amirafshar, Jude Haris, Nima Taherinejad, José Cano
publishedcode/data similarity 0.66
Abstract · Accepted at the 38th IEEE/SBC International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD) 2026

Approximate multipliers can reduce hardware area and energy consumption in Deep Neural Network (DNN) inference; however, they introduce computational errors. Assessing the accuracy of numerous approximate multiplier designs across diverse DNN models and large-scale datasets remains challenging due to prohibitive evaluation times. This overhead primarily stems from the slow emulation of approximate multiplier behavior using look-up tables (LUTs) on CPU and GPU platforms. Moreover, the resulting accuracy degradation must be carefully quantified and, if necessary, mitigated (e.g., through retraining), further increasing the overall evaluation cost. To address these challenges, we propose FAME, an FPGA-based platform for evaluating approximate multipliers. The platform exploits the reconfigurable logic of Field-Programmable Gate Arrays (FPGAs) to implement approximate multipliers directly in hardware, eliminating the need for LUT-based emulation on CPU/GPU platforms and thereby enabling efficient DNN inference while significantly reducing evaluation time on large datasets. Furthermore, we introduce a pattern-guided DNN retraining technique to mitigate accuracy degradation induced by approximate multipliers. Specifically, retraining is guided by multiplier-specific patterns to effectively recover potential accuracy losses. We evaluate FAME using two DNN models, ResNet-18 and MobileNetV2, on the ImageNet dataset across 27 approximate multipliers. During inference, our approach achieves up to a 3.47x speedup in approximate multiplier evaluation compared to prior LUT-based emulation methods. Furthermore, the proposed retraining technique improves accuracy by up to 65.5% over existing retraining approaches for the evaluated multipliers. The code is publicly available at: https://github.com/gicLAB/FAME

0.28

LEAF: LUT-aware Efficient Attention on FPGAs

cs.LG hep-ex ·2026-10-19 ·Arianna Cox, Chang Sun, Lauri Laatu, Benedikt Maier, Aadeesh Sharma +5 more
publishedbenchmark similarity 0.66
Abstract · Accepted for the 9th Workshop on Machine Learning and the Physical Sciences (ML4PS 2026)

We introduce Lookup-table (LUT)-aware Efficient Attention on FPGAs (LEAF). LEAF is the first nanosecond-scale LUT-aware transformer for FPGAs. Deploying transformers that are large enough to perform well, small enough to fit on an FPGA, and fast enough to meet the sub-microsecond timescales required by real-time systems such as those at CERN remains difficult, since the same attention mechanism that gives these models their high expressivity also makes them costly to implement in hardware. LEAF converts the query, key, and value projections and the final output feed-forward layer into FPGA-native layers with LUT-aware training, creating the first LUT-aware transformer and linformer layers built for ultra-low-latency inference. Attention resists LUT-aware training because it multiplies two activations where the method needs a constant weight to tabulate; LEAF tabulates the attention block's constant matrix-vector multiplications and leaves the rest in arithmetic form. On a standard jet tagging benchmark this improves accuracy by 1.8 percentage points while using 2.9 times fewer LUTs than the previous state-of-the-art sub-microsecond transformer at the same input size. Our most efficient LEAF-based linformer with 64 input particles reaches 80.4% classification accuracy at 28k LUTs and 35ns.

0.34

Energy-Efficient FPGA Implementation of an End-to-End Cosine-Domain CNN Using High-Level Synthesis

cs.AR ·2026-10-29 ·Sana Fatima, Ab Al-Hadi Ab Rahman, Ibrahim Yousef Alshareef, Farhad Khodadadi, Nuzhat Khan
similarity 0.65
Abstract · 9 pages, 11 figures. Currently under review at the VLSI Integration Journal

The computational cost of conventional Convolutional Neural Networks (CNNs) limits their deployment in low-power devices for high-speed edge AI applications. This paper presents an end-to-end cosine-domain CNN (CD-CNN) architecture for energy-efficient FPGA inference using high-level synthesis (HLS). The proposed architecture performs intermediate feature extraction using real-valued DCT-domain representations, avoiding complex-valued arithmetic and intermediate inverse transforms between spectral layers. The design is implemented on a Xilinx Kintex UltraScale+ XCKU5P FPGA and evaluated against a fully spectral Fourier-domain CNN (FD-CNN) under consistent network and implementation constraints. Experiments on the MNIST and 94-class ASCII datasets show that the CD-CNN reduces normalized resource utilization and improves inference throughput on both datasets. Its energy efficiency reaches 112.91 frames/J for MNIST and 101.27 frames/J for ASCII, corresponding to improvements of $1.51\times$ and $2.76\times$, respectively, over the fully spectral FD-CNN architecture. Furthermore, compared to the previous GPU-based fully spectral CD-CNN implementation, the proposed end-to-end FPGA implementation improves energy efficiency by $12.96\times$. These results demonstrate the potential of real-valued spectral processing in the DCT domain for energy-efficient edge AI deployment.

0.28