CS Paper Radar

CNN-Transformer with Sigmoid Label for High-Resolution Sub-Degree DOA Estimation

Lifeng Wang, Xintong Li, Aifei Liu

eess.SP cs.AI ·submitted 2026-10-09 11:34 UTC · v1

Abstract

This paper addresses the grid quantization error of deep-learning (DL)-based direction-of-arrival (DOA) estimation. Classification- based DOA methods discretize the continuous angular space into a grid, so that with a peak-based read-out, the accuracy and resolution are limited by the grid step and by the width of the label peak. To address this issue, we propose a sigmoid label that encodes the sources' DOAs as the sum of angle-shifted monotonic sigmoid functions. Differentiating the output of a CNN-Transformer network trained with the sigmoid label produces sharp spectral peaks at the source DOAs. Combined with a moderately refined grid step of 0.25 degrees, the CNN-Transformer with the proposed sigmoid label attains sub-degree accuracy and a 1-degree resolution, breaking the limit of the grid quantization error. Under the unified CNN-Transformer backbone, we compare the proposed Sigmoid label with the Gaussian label, the 0-1 hard label, and the binary cross-entropy (BCE) baseline. Numerical results demonstrate that the Sigmoid label attains the highest accuracy and superior resolution compared to other labels.

Closest papers

Nearest neighbours by meaning across the whole index.

Quantized Transformers for Massive MIMO Precoding with Automatic Resolution Tuning

eess.SP ·2026-10-03 ·Ghazal Kasalaee, Glodi Sala Mangituka, Ali Hasanzadeh Karkan, Jean-François Frigon, François Leduc-Primeau
similarity 0.61
Abstract · IEEE International Workshop on Signal Processing Systems (SiPS)

Deep learning precoders, particularly those based on Transformer architectures, offer superior spectral efficiency for Massive MIMO systems but may incur high computational cost. While existing mixed-precision quantization studies on convolutional neural networks establish important energy baselines, traditional discrete search methods fail to scale to the large parameter spaces of modern Transformers. To bridge this gap, we apply a differentiable precision learning technique to massive MIMO precoding. This approach autonomously and jointly optimizes weights, activations, quantization step sizes, and layer-wise bit-widths in a single training loop, directly optimizing for energy efficiency. We further expand this approach by incorporating Neural Architecture Search across various Transformer model sizes and evaluating the impact of random versus floating-point trained initializations on resolution tuning. Ultimately, we demonstrate that the proposed training method enables the deployment of highly compact Transformer precoders, improving energy efficiency by up to 288\(\times\) compared to the classical Weighted Minimum Mean Square Error algorithm at equal sum-rate performance in a dense downtown environment.

0.26

Energy-Efficient Visual Inspection with FFT-Based CNNs and Adaptive Floating-Point Quantization

cs.AR ·2026-08-20 ·Lukas Krupp, Marco Groß, Michael Graichen, Kim Ulrich, Norbert Wehn
published similarity 0.56
Abstract · Accepted for presentation at the 23rd International SoC Design Conference (ISOCC 2026). Proceedings to be included in IEEE Xplore

This paper investigates reduced-precision floating-point arithmetic for FFT-based CNN inference on an industrial CPU-FPGA platform. We combine FFT-based convolution with adaptive post-training FP8 quantization and evaluate two FPGA-oriented optimization methods: progressive bias adjustment (PBA) within the FFT and layer-wise exponent-bias selection across the CNN. The methods are implemented in a LeNet-5 accelerator using serial radix-$2^2$ SDF FFT modules and evaluated on an industrial fault detection dataset. Results show that weight scaling outperforms PBA, while layer-wise bias optimization increases the accuracy from 80.33% to 84.13% without modifying the datapath width. Compared with CPU-only inference, the FPGA achieves approximately 2.5$\times$ higher energy efficiency.

0.44

Energy-Efficient FPGA Implementation of an End-to-End Cosine-Domain CNN Using High-Level Synthesis

cs.AR ·2026-10-29 ·Sana Fatima, Ab Al-Hadi Ab Rahman, Ibrahim Yousef Alshareef, Farhad Khodadadi, Nuzhat Khan
similarity 0.55
Abstract · 9 pages, 11 figures. Currently under review at the VLSI Integration Journal

The computational cost of conventional Convolutional Neural Networks (CNNs) limits their deployment in low-power devices for high-speed edge AI applications. This paper presents an end-to-end cosine-domain CNN (CD-CNN) architecture for energy-efficient FPGA inference using high-level synthesis (HLS). The proposed architecture performs intermediate feature extraction using real-valued DCT-domain representations, avoiding complex-valued arithmetic and intermediate inverse transforms between spectral layers. The design is implemented on a Xilinx Kintex UltraScale+ XCKU5P FPGA and evaluated against a fully spectral Fourier-domain CNN (FD-CNN) under consistent network and implementation constraints. Experiments on the MNIST and 94-class ASCII datasets show that the CD-CNN reduces normalized resource utilization and improves inference throughput on both datasets. Its energy efficiency reaches 112.91 frames/J for MNIST and 101.27 frames/J for ASCII, corresponding to improvements of $1.51\times$ and $2.76\times$, respectively, over the fully spectral FD-CNN architecture. Furthermore, compared to the previous GPU-based fully spectral CD-CNN implementation, the proposed end-to-end FPGA implementation improves energy efficiency by $12.96\times$. These results demonstrate the potential of real-valued spectral processing in the DCT domain for energy-efficient edge AI deployment.

0.28

Comparative Evaluation of Deep-Learning based DoA Estimators for Real-Time ISAC Beamforming

cs.LG ·2026-10-14 ·Anjali K. S., Manoj Kumar Panda, Chandramani Singh
similarity 0.55
Abstract

Integrated Sensing and Communication (ISAC) systems rely on accurate Direction-of-Arrival (DoA) estimation for effective beamforming and spectral-efficiency enhancement. While numerous classical and deep-learning-based DoA estimation methods have been reported in the literature, their suitability for real-time ISAC operation remains insufficiently understood because practical deployment requires consideration of both estimation accuracy and computational latency. This paper presents a unified evaluation framework for systematically comparing classical and learning-based DoA estimators within an ISAC beamforming pipeline. Representative classical and learning-based DoA estimators, together with an ensemble architecture, are integrated with Singular Value Decomposition (SVD)-based beamforming and evaluated under identical operating conditions. The impact of DoA estimation accuracy, inference latency, beamforming-weight generation time, and resulting effective spectral efficiency is investigated. Results show that different estimators occupy distinct operating points in the accuracy-latency-performance design space. Although learning-based approaches substantially reduce estimation time compared with MUSIC, the overall processing latency increasingly becomes dominated by the computational cost of SVD-based beamforming. Among the evaluated methods, the Regression Convolutional Neural Network (RCNN) exhibits a favorable balance between estimation accuracy, computational efficiency, and beamforming performance, making it a promising candidate for real-time ISAC deployment. The findings further suggest that future low-latency ISAC systems may benefit from beamforming strategies that reduce or avoid computationally intensive SVD-based beamforming-weight generation.

0.20

ProgResViT: Progressive Resolution and Width for Adaptive Vision Transformers

cs.CV ·2026-09-02 ·Ali Hojjat, Janek Haberer, Olaf Landsiedel
code/data similarity 0.55
Abstract

Vision Transformers (ViTs) typically process every image using a fixed input resolution and model width, even though many images can be classified with substantially less computation. We introduce ProgResViT, an input-adaptive ViT that performs inference progressively across multiple rounds. The first round processes a low-resolution image with a narrow subnetwork. Inference terminates when the prediction is sufficiently confident; otherwise, the model reuses the representations produced in the current round and proceeds with a higher-resolution input and a wider subnetwork to refine its prediction. As all rounds share a single backbone, we propose Progress-Conditioned Soft Gating (PSG), which conditions token fusion and layer outputs on the current round, block, and input resolution. On image classification, applying ProgResViT to DeiT yields better accuracy-compute trade-offs than adaptive-width, adaptive-depth, and dynamic-token baselines. With knowledge distillation, a DeiT-based ProgResViT achieves 84.9% top-1 accuracy, slightly exceeding the reported DeiT-III-S accuracy under a comparable evaluation setting. We show that the same design also provides favorable accuracy-compute trade-offs for self-supervised DINO representations and downstream semantic segmentation. Code is available at https://github.com/ds-kiel/ProgResViT.

0.33

Factorized axis convolutional gated recurrent unit with dynamic adaptive pooling for remaining useful life prediction of rolling bearings

cs.AI ·2026-09-25 ·Hanbyeol Park, Jungho Choo, Hyerim Bae
similarity 0.55
Abstract

Convolutional neural networks (CNN) are widely used to predict the remaining useful life (RUL) of rolling bearings from time-frequency representations (TFRs) of vibration signals. However, during degradation, characteristic structures in TFRs align predominantly along the frequency or time axis, making it challenging for conventional CNN isotropic kernels to capture directional structure. Furthermore, global average pooling (GAP) averages across axes, potentially obscuring the locations and concentrations of salient activations. This study introduces a factorized-axis convolutional gated recurrent unit (GRU) that employs multiscale anisotropic convolution and dual-axis convolution block attention module to enhance directional features and highlight salient time-frequency regions. Dynamic adaptive pooling (DAP) adaptively aggregates the time-frequency-axis information from the extracted feature maps, whereas a GRU captures temporal dynamics in the latent representations and Monte Carlo dropout enables predictive uncertainty estimation. Experiments on two public bearing datasets demonstrate that the proposed model outperforms existing RUL prediction methods across operating conditions. Ablation experiments demonstrate that the factorized axis-wise design achieves lower mean errors than convolutional isotropic kernels. DAP yields clear improvements on one dataset while matching GAP on the other, highlighting the importance of anisotropic feature extraction and adaptive feature aggregation for TFR-based RUL prediction.

0.21

Transformers as Generic Deep Neural Networks: Exact Compilation, the Cost of Removing Activation-Activation Products, and Multi-Agent Attention Clustering

cs.LG cs.MA ·2026-10-20 ·Sergey Salishev, Zeev Volkovich, Oleg Granichin
theorycode/data similarity 0.55
Abstract · 36 pages, 6 tables and 2 figures in the main text, 8 supplementary tables. Code: https://github.com/iceslice0/transformer-to-dnn

Transformers are deployed as a special model class. Neural processing units (NPU) add dedicated operators for attention and normalization, and quantization relies on Transformer-specific algorithms. We show the computation does not force this. Every Transformer block, with attention, LayerNorm and a GELU MLP, compiles exactly into a generic deep neural network built from fixed affine maps, routing and the square, sigmoid and inverse square root, plus GELU or its sigmoid approximation. Without activation-activation products, every transform-wise compilation adds at least $N^2$ scalar nonlinear evaluations per head for dense scores. Keeping that one operation type removes this cost; its elementwise form, present on every surveyed NPU, suffices with the sigmoid and inverse square root. For top-$k$ attention with exchangeable tail scores, we prove that exact or conditional-mean tail mass gives an unbiased, minimum-mean-square-error estimate of the attention distribution, while tail drop is conditionally biased. When queries or agents cluster, centroids choose the keys first, reducing the number of computed scores from $N^2$ to $Nk$ per head. On DeiT-Tiny, MambaIRv2-Light and Pythia-70M the compiled models match their references with all keys, standard post-training quantization applies, and the exact tail-mass estimate beats tail dropping with half as many keys on DeiT-Tiny and, for $k\le8$, on Pythia-70M.

0.27

MCL: Meta Convolution Layer

cs.CV ·2026-10-08 ·Naim Reza, Md Al Amin, Ho Yub Jung
similarity 0.54
Abstract

Dynamic convolution enhances convolutional neural networks (CNNs) by adapting kernels to input content, but it expresses the effective kernel as a linear mixture of a small number of basis kernels, which limits expressivity and complicates optimization as the mixture size grows. In this work, we revisit dynamic convolution from a functional perspective and propose the Meta Convolution Layer (MCL), which directly models the convolutional kernel as an input-conditioned function W(x) realized via a high-order polynomial expansion. Leveraging nested residual blocks inspired by deep polynomial networks, MCL implements a structured polynomial meta-network that generates a single input-adaptive kernel, thereby decoupling representational power from the explicit number of mixture kernels and alleviating training instability. MCL is a plug-in addition with standard convolutions and can be seamlessly integrated into both CNN and transformer backbones. Experimental evaluation shows that adding MCL improves the Top-1 accuracy of Resnet- 18, Resnet-50 and ResNet-101 by 6.61%, 3.42% and 3.05% on the ImageNet dataset. Moreover, the proposed method significantly boosts the accuracy of Resnet and Wide-Resnet variants on CIFAR-10 and CIFAR-100 datasets. Additionally, the proposed method outperforms previous methods on fine-grained visual classification tasks using Swin and ViT backbones. These results demonstrate that high-order polynomial kernel generation is a powerful and scalable alternative to linear mixture based dynamic convolution.

0.14