CS Paper Radar

GaLe: memory-efficient Global Approximate and Local Exact features

Alberto Ancilotto, Elisabetta Farella

cs.CV ·submitted 2026-09-02 14:56 UTC · v1
published

Abstract

Embedded devices typically lack the resources of GPU-equipped machines, and existing inference methods suffer from either high computational overhead (patch-based) or accuracy loss (approximation-based). We propose GaLe, a memory-efficient technique that enables the deployment of pretrained networks on constrained devices without retraining. GaLe partitions feature maps into two components: a local exact (Le) representation that preserves fine details and a global approximate (Ga) representation that retains long-range dependencies. Unlike standard tiling, GaLe supports global operations and attention mechanisms found in hybrid CNN-transformer models. Validated on ImageNet, our method matches exact-inference performance while achieving up to 65% speedup and 90% RAM reduction on a Cortex-M33 compared to patch-based inference. We further demonstrate GaLe's versatility across classification, detection, and generation tasks, highlighting its potential as a foundation for resource-efficient architecture design.

Closest papers

Nearest neighbours by meaning across the whole index.

Node-wise Feature Encoding for Neural Performance Prediction

cs.LG ·2026-08-28 ·Matthew Grenier, William Hammer, Andrew Heuer, Nikhil Krishna, Yi Wang +1 more
similarity 0.72
Abstract · 22 pages, 7 figures

As neural networks are increasingly deployed on resource constrained edge devices, accurate prediction of latency and energy is critical for efficient neural architecture search. Existing GNN and transformer based predictors achieve strong results but largely ignore node-level computational cost, limiting their ability to model performance critical operations. To address this, we introduce FeatureFormer, a neural performance predictor that incorporates explicit node-wise encodings of FLOPs, parameter counts, and memory proxies within a gated graph attention architecture. We also present NNEQ, a new large-scale energy consumption dataset that enables unified evaluation of latency and energy prediction. Extensive experiments demonstrate that FeatureFormer achieves state-of-the-art performance across both metrics, including challenging out-of-domain settings. Finally, we show that the proposed encoding is broadly applicable and consistently improves existing predictors with negligible overhead.

0.13

Sub-100KB On-Device CNN Training via Dynamic Sparse Updates and Hardware--Software Co-Design

cs.AR ·2026-11-11 ·I-Hsuan Li, Tian-Sheuan Chang
top venue similarity 0.72
Abstract · published in AICAS 2026

The increasing demand for real-time processing and personalized deep learning has made edge-device training essential but challenging due to high memory and computational costs. To address this, we propose a Convolutional Neural Network (CNN) training accelerator with a dynamic sparse update mechanism that selectively updates important parameters using a random channel strategy, avoiding costly evaluations. Combined with activation pruning, our approach reduces activation storage to just 3.4\% of the original. Our hardware features a fused forward dataflow that concurrently processes group normalization and convolution, reducing training time by 45\% through computation reordering and efficient zero-skipping. Implemented in TSMC 28nm at FP8 precision, the design operates at 800 MHz, achieving 8 TOPS peak throughput, 31.13 TOPS/W energy efficiency, and 14.8 TOPS/mm$^2$ area efficiency, with power consumption ranging from 257 to 272 mW. The design enables on-device training within a sub-100KB on-chip SRAM budget (95KB in silicon).

0.36

FoveaDet: Software-Hardware Co-Design of Foveated CNN-Transformer for Object Detection

cs.CV ·2026-11-11 ·Deepak Mewada, Apporva Saraf, Ashwin Lele
similarity 0.71
Abstract · 5 pages, 6 figures. Submitted to IEEE ISCAS 2027

Deploying real-time object detection on power-constrained edge platforms is challenging as processing full-resolution frames is often computationally redundant. Inspired by biological foveation which assigns variable resolution based on visual saliency, we propose a novel software-hardware co-design framework called FoveaDet. On the software side, FoveaDet dynamically identifies salient regions to predict fixation points, followed by aggressive patch pruning in Vision Transformers (ViT) to discard irrelevant background data. To mitigate slowdowns from sequential patch generation, we implement optimized interpolation and patch selection algorithms. Evaluated on standard GPUs, this foveated approach reduces floating-point operations by 65.4% to 92.2% and accelerates inference by 4.76x, while suffering a negligible mAP drop compared to dense baselines. To eliminate hardware bottlenecks, we complement the algorithm with a custom RISC-V core featuring specialized instructions to accelerate the foveated processing pipeline. FoveaDet provides a scalable, efficient architecture capable of unlocking advanced vision processing on severely constrained mobile platforms.

0.26

From Deep to Shallow: Unconstrained and Efficient Layer Merging Strategy

cs.LG ·2026-09-04 ·Petro Shulzhenko, Gabriele Spadaro, Enzo Tartaglione
code/data similarity 0.70
Abstract · 13 pages, 3 figures, accepted at the ITEM Workshop at ECML PKDD 2026

Although Deep Neural Networks have become foundational in many areas of Machine Learning, high computational demands limit their application in resource-constrained environments. To address this issue, depth compression methods have been proposed to identify and linearize redundant activation functions, thereby allowing for the merging of layers without intermediate non-linearities. However, these methods face two key challenges: they cannot be directly applied to convolutions with padding due to the absence of an analytical solution for merging these layers, and they typically increase the kernel size of merged layers, thus limiting speed-up gains. To overcome these limitations, we propose an efficient strategy that enables merging of layers without an existing analytical solution, and also without increasing kernel size. We validate our approach across multiple architectures and datasets, and measure inference speed-up gains on real embedded platforms. We publicly released the code at https://github.com/ShulzhenkoPetr/deep-to-shallow.

0.36

A Hardware-Aware Comprehensive Study of Edge Vision Deployment with A Unified Hardware-Model-Runtime Mapping Framework

cs.CV ·2026-11-18 ·Le Duc An, Le Duc Thanh, Vo Hoai Viet
surveybenchmark similarity 0.70
Abstract

The rapid growth of deep learning has driven the deployment of computer vision directly on resource-constrained edge devices. However, the extreme heterogeneity of edge hardware architectures poses major challenges in selecting and designing suitable vision models. This survey bridges the gap between edge hardware constraints, model architectures, and deployment software toolchains. We establish a comprehensive taxonomy classifying edge hardware into three distinct tiers: mid-range devices (GPUs and dedicated NPUs or TPUs), dedicated embedded MPU or SoC processors, and microcontrollers (MCUs). We aggregate and analyze multi-source benchmark results for image classification and object detection across the computing spectrum, spanning from tens of watts down to milliwatt-scale bare-metal MCUs. Based on these empirical findings, we propose a two-tiered hardware-to-model mapping framework that provides concrete model selection matrices across performance priorities (Speed, Balanced, Accuracy) and actionable architectural design principles (peak SRAM budgeting, resolution-first scaling, and algorithm-compiler co-design). Finally, we highlight critical deployment challenges including toolchain fragmentation, operator incompatibility, and Vision Transformer feasibility, and outline promising future research directions in on-device adaptation and sustainable Edge AI.

0.19

RAMP: Robust Adaptive Mixed-Precision Quantization for Edge CPU Vision Models

cs.CV cs.LGcs.PF ·2026-09-23 ·David Población-Criado, Dario Garcia-Gasulla, Eduardo Quinones
publishedcode/data similarity 0.69
Abstract · Accepted at the 37th British Machine Vision Conference (BMVC) 2026. 13 pages, 4 figures, 2 tables. Code available at https://github.com/davi…

Deploying deep learning models on edge CPUs is bottlenecked by computational and memory constraints. Mixed-precision quantization promises to reduce inference latency while preserving accuracy. However, quantization affects different layer types in inconsistent ways, so identifying where accuracy loss is minimized and latency reduction is maximized is critical, as the effect accumulates over a full deployment into substantial savings or unacceptable task degradation. Such identification relies on sensitivity metrics, proxies that estimate layer-wise degradation without evaluating the task accuracy of every candidate policy. Nevertheless, widely used metrics fail systematically on modern architectures. We present a systematic empirical study of 13 sensitivity metrics for layer-wise INT8 quantization across four distinctly different neural networks, and validate the resulting policies on two ARM64 platforms. Gradient-based sensitivity methods fail on 4 out of 8 model-hardware configurations and weight-based statistics on 2. In contrast, the Jensen-Shannon Divergence achieves zero catastrophic failures, reliably isolating the layers that cannot be safely quantized. A sensitivity metric alone does not define a policy, and the fixed thresholds typically used for that step are fragile over the highly skewed distributions of modern architectures. We address this with K-Means clustering, achieving near-lossless accuracy and a mean speed-up of $1.81\times$ over the full-precision model. Finally, we reveal that excluding from quantization the layers whose speed-up is negligible, regardless of their sensitivity, can be counterproductive, as it induces computational graph fragmentation and disables operator fusion. Our results yield concrete allocation policies for practitioners and researchers deploying quantized vision models on heterogeneous edge CPUs, without GPU access or gradient computation.

0.51

Rethinking Data Efficiency in Industrial Dense Prediction: Pretraining Coherence, Not Inductive Bias, Determines ViTs Low-Data Advantage

cs.CV ·2026-08-11 ·Haoran Sui, Yaoyuan Jia
similarity 0.69
Abstract · 14 pages, 10 figures, 17 tables. This paper targets industrial defect detection via vision transformer and CNN alignment grafting

Vision Transformers (ViTs) are widely believed to require more labeled data than CNNs for industrial dense prediction. Through controlled experiments on four industrial datasets, we show that the data-efficiency gap stems from pretraining incoherence, which refers to the statistical mismatch between ImageNet-pretrained ViT backbones and COCO-pretrained CNN necks, rather than from inherent self-attention deficits. We characterize the cross-architecture feature gap and propose a lightweight AlignBlock family for pyramid-level feature recalibration. Our core finding empirically identifies a data-efficiency frontier: for domain-proximal scenes with >= 200 samples, Swin-Graft surpasses YOLOv11x (terminal 703-shot: 0.973 vs 0.956 mAP@50); for domain-distant scenes, CNNs retain advantage (hook 141-shot: 0.900 vs 0.600 mAP@50). Grafted neck weights yield up to 2.5x the mAP of a randomly initialized neck.

0.10

CARB: A Characterization-Guided Framework for CNN Inference Cost Prediction and Deployment Screening

cs.AR cs.LGcs.PF ·2026-08-11 ·Linh Nguyen, Zhixin Pan
similarity 0.69
Abstract

Accurate pre-deployment estimation of CNN inference cost--energy, latency, and peak memory--is increasingly critical as models are deployed on resource-constrained GPU platforms. Existing approaches rely on FLOPs, latency measurements, or single-device profiling as energy proxies, overlooking the non-linear interactions between architectural design and hardware load. We present a workload characterization study of 13 419 CNN configurations on two GPU platforms (RTX 5090 and RTX 3080) under GPU telemetry, revealing that energy, latency, and memory exhibit fundamentally distinct scaling behaviors: energy and latency diverge by 3x under high computational demand, and cross-GPU transferability differs by target--energy and latency require platform-specific models while memory transfers well across the two tested platforms. Building on these characterization findings, we develop CARB, a cascade-blended ensemble that jointly predicts all three targets with R2 ~0.99, and a two-stage deployment screening workflow that eliminates over 90% of candidates in seconds, reducing large design spaces to a Pareto-prioritized shortlist validated against real hardware.

0.04