CS Paper Radar

On the Instance Hardness as a Decision Criterion in TinyML Systems

Tobiasz Puslecki, Krzysztof Walkowiak

cs.AI cs.LG ·submitted 2026-08-30 17:18 UTC · v1

Abstract

TinyML includes the implementation of machine learning on devices with limited memory and computing resources. With the development of technology, AI systems continue to scale in terms of size and computational requirements. This forces researchers to adapt methods to be environmentally sustainable by designing techniques for reducing computational costs and energy consumption in inferring AI models, even in small devices. In this work, we present preliminary findings on a novel application of the tree depth prune instance hardness method to the TinyML system. The results indicate that threshold control can change energy consumption with limited classification quality changes. This method allows us to adjust classification accuracy, thereby influencing computational complexity and energy consumption for inference. We present a work in progress with initial results as a proof of concept.

Closest papers

Nearest neighbours by meaning across the whole index.

Power-Performance Characterization of TinyML Systems

cs.LG cs.AI ·2026-08-21 ·Yujie Zhang, Dhananjaya Wijerathne, Zhaoying Li, Tulika Mitra
publishedhighly cited similarity 0.66
Abstract · 7 pages, 9 figures. Published in Proceedings of the IEEE International Conference on Computer Design (ICCD), 2022

TinyML systems are enabling machine learning (ML) inference at the edge. However, there is little quantitative analysis of such systems. This paper presents a systematic performance and power characterization of diverse TinyML applications on microcontrollers (MCUs), spanning neural network models, software libraries, operating systems, and hardware architectures. We focus on the impact of the multiple layers of abstraction that provide higher programmability at the expense of performance and energy efficiency. We propose a model to estimate the costs of different abstraction layers and make recommendations for minimizing those costs. Our findings can help designers with Neural Architecture Search (NAS) and CNN inference optimization on edge devices.

0.54

Energy Measurement and Optimization of TinyML in Resource-Constrained Embedded Systems: A Systematic Mapping Study

cs.LG cs.ARcs.ET ·2026-11-05 ·Clariele Almeida, Renan Beserra, Rafael Moura, Danilo Araújo, Bruno Nogueira +2 more
survey similarity 0.60
Abstract · 40 pages, 10 figures, 8 tables

The rapid expansion of the Internet of Things (IoT) has increased the volume of data generated at the network edge, where many devices operate under severe memory, processing, and energy constraints. In this context, Tiny Machine Learning (TinyML) enables machine learning (ML) tasks to be executed on microcontrollers, smart sensors, and other resource-constrained platforms. However, the literature remains heterogeneous regarding the algorithms evaluated, energy-measurement methods, reported metrics, influencing factors, and optimization strategies. This study presents a systematic mapping of energy-aware TinyML research. It identifies the main models and learning approaches, classifies energy-evaluation methodologies and metrics, examines the technical factors affecting energy consumption, and organizes the reported optimization techniques into a structured taxonomy. The findings highlight the frequent use of convolutional and lightweight architectures, direct measurements and analytical estimations, and factors related to model architecture, hardware, communication, numerical precision, and input characteristics. The optimization strategies were grouped into model compression, hardware-aware design, system and communication optimization, and data and processing optimization. The results also reveal limited standardization in experimental platforms, measurement procedures, and reporting practices, limiting reproducibility and direct comparison across studies. This mapping provides an updated overview of the field and guidance for developing and evaluating more energy-efficient TinyML solutions.

0.25

TinyML Hits the Memory Wall: A Benchmark of Ten Sub-Watt Edge Devices

cs.AR ·2026-11-02 ·Pietro Bonazzi, Luigi Capogrosso, Ahmet Celik, Rosanna Manzo, Marco Giordano +3 more
benchmark similarity 0.54
Abstract

Neural network inference at the edge must fit tight energy budgets. Existing benchmarks compare few devices, do not report energy on a common basis, and omit the newest microcontrollers with an NPU or the 1 GHz Cortex-M85. Hardware-aware design also ranks networks by MAC count, yet no study has tested whether this proxy holds across accelerators. We benchmark sub-watt edge AI hardware by evaluating four CNNs (MobileNetV3, MCUNetV1, EfficientNet, SqueezeNet) and two tiny transformers (MobileViT-XXS, DeiT-tiny) on ten commercial devices: an in-sensor DSP (Sony IMX500), four MCUs with an NPU (STM32N6, Alif Ensemble E7/E8, GreenWaves GAP9), and five MCUs without an NPU (STM32U5, Apollo510B, ESP32-S3 Xtensa LX7, RA8M2, STM32V8). Four devices are novel (Alif E8, Apollo510B, RA8M2, STM32V8). Measured as energy above idle, the IMX500 (0.13-0.35 mJ) uses 1.3-3x less energy than the STM32N6 (0.31-1.05 mJ) on every model; the GAP9 uses the least on MobileNetV3 and the Alif E7 the least on MCUNetV1 and SqueezeNet. All four use 6-34x less energy than the STM32V8 (1.85-3.98 mJ), and the Alif E7 and IMX500 form the power-performance Pareto front. Latency on all four accelerators scales with memory traffic (bytes of weights and activations, r >= 0.98) and not with MACs, so they are memory-bound, while no MCU without an NPU is. The NPU advantage grows with intensity from 1.4x over a 1 GHz Cortex-M85 on MCUNetV1 to 8.2x on SqueezeNet. MAC count is therefore a misleading cost proxy for sub-watt accelerators.

0.05

SAT-based Encodings for Optimal Decision Trees with Explicit Paths

cs.LO ·2026-08-20 ·Mikoláš Janota, António Morgado
top venuehighly cited similarity 0.53
Abstract

Decision trees play an important role both in Machine Learning and Knowledge Representation. They are attractive due to their immediate interpretability. In the spirit of Occam's razor, and interpretability, it is desirable to calculate the smallest tree. This, however, has proven to be a challenging task and greedy approaches are typically used to learn trees in practice. Nevertheless, recent work showed that by the use of SAT solvers one may calculate the optimal size tree for real-world benchmarks. This paper proposes a novel SAT-based encoding that explicitly models paths in the tree, which enables us to control the tree's depth as well as size. At the level of individual SAT calls, we investigate splitting the search space into tree topologies. Our tool outperforms the existing implementation. But also, the experimental results show that minimizing the depth first and then minimizing the number of nodes enables solving a larger set of instances.

0.68

Reducing Inference Cost of Coding Agents with Timeout Cascades

cs.AI cs.SE ·2026-11-18 ·Boxi Zhou, German Sviridov, Dmitrii Ustiugov, Haris Javaid
benchmarkcode/data similarity 0.52
Abstract

AI coding agents have become one of the most popular applications of large language models. Yet, agentic AI comes at the cost of a tremendous amount of GPU compute resources, which limits its profitability and its viability at scale. In this paper, we investigate whether smaller, cheaper models can cut this cost without sacrificing accuracy. We evaluate 36 popular large and small open-weight models released since 2024 on a software engineering benchmark, and find that recent small models are closing the accuracy gap compared to their large counterparts. We investigate the structure of this gap and find that the gap is highly concentrated around hard tasks that small language models fail to solve while achieving remarkably good accuracy for easy and medium complexity tasks. Following this observation, we show that by utilizing a hybrid inference strategy that tries a small model first and escalates to a large one only when needed, we can achieve up to 33% reduction in total GPU-hour costs and 16.4% on all tasks from SWE-bench Verified, without sacrificing accuracy. Strikingly, this simple cascade nearly matches an oracle-style task router, highlighting a practical path for deploying cost-aware coding agents in production without complex learned task routers and with minimal engineering complexity.

0.29

The thermodynamic freedom of a thermodynamic computer

cs.NE cond-mat.stat-mech ·2026-08-28 ·Stephen Whitelam
similarity 0.51
Abstract

Thermodynamic computers are stochastic physical devices designed to perform calculations at the thermal energy scale. Their operation is constrained by the equations of stochastic thermodynamics, among which are a set of bounds, known as speed limits, that relate a thermodynamic computer's run time to its computational progress and the heat it dissipates. Using the Wasserstein speed limit we assess the thermodynamic efficiency of a simulation model of a thermodynamic computer trained to perform a standard machine-learning classification task. On this task the thermodynamic computer is as capable as a simple multilayer perceptron. We show that different inference protocols allow the computer to operate within 40\% of the thermodynamic limit of efficiency without loss of accuracy, or to perform inference increasingly rapidly at fixed accuracy and thermodynamic efficiency. These results indicate that a thermodynamic computer designed for a particular task retains considerable freedom in its thermodynamic operation.

0.25

PolyChirp: Multi-Species Birdsong Classification Using TinyML on Low-Power Acoustic Sensors

cs.LG cs.AI ·2026-08-24 ·Nathan Duboisset, Zhaolan Huang, Felix Bießmann, Roudy Dagher, Antoine Lavandier +1 more
published similarity 0.51
Abstract

Recent progress in the field of TinyML has demonstrated that low-power hardware based on microcontrollers can achieve bird species monitoring in real time based on acoustic sensor data for an entire breeding period on a single battery charge. However, the state of the art on low-power microcontrollers was so far limited to binary classification of a single species. In contrast, real fauna monitoring deployments often target multiple species simultaneously. To address this challenge we develop PolyChirp, an approach combining biological domain expertise, automated dataset curation, neural architecture optimization and novel hardware to achieve multiclass bird species detection in the wild. PolyChirp is based on newly designed tiny multiclass models that leverage recent microcontrollers and hardware acceleration with a neural processing unit (NPU). We evaluate the predictive performance of these models, and we measure their computational performance -- flash footprint, latency, energy consumption -- on common microcontroller hardware. Our results demonstrate that PolyChirp matches or exceeds the TinyChirp architectures retrained under our protocol on single-species detection, and further achieves robust classification of up to 10 species simultaneously (macro F2 up to 0.97), while still fitting the flash, latency and energy budget of a low-power microcontroller sensor. A data-driven front-end redesign additionally makes on-device mel feature extraction 7x to 11x cheaper.

0.38

PELM: Power Efficient On-Device LLM Inference with Speculative Decoding and Dynamic Voltage Frequency Scaling

cs.LG cs.CLcs.OScs.PF ·2026-09-09 ·Weisi Yang, Stephen Xia
publishedcode/datahighly cited similarity 0.50
Abstract · Accepted to ACM/IEEE SenSys'26

Deploying Large Language Models (LLMs) directly on mobile platforms at the edge is gaining traction due to a myriad of benefits, such as increased privacy, personalization, and reduced latency. However, LLMs have heavy computational requirements, which are difficult for resource-constrained mobile and edge platforms to fulfill. In addition to limited compute resources, mobile and edge systems often have a compact form factor and lack physical mechanisms to dissipate heat generated from high processor usage rates (e.g., fans) to prevent throttling and reduced processing power, which LLMs can easily cause. To mitigate these effects, prior works have proposed various power governing strategies, such as dynamic voltage and frequency scaling (DVFS), for reducing power and heat generation for heavy computational tasks on mobile platforms. Recently, DVFS methods tailored for mobile LLMs have also been proposed. However, these methods mostly focus on optimizing hardware parameters and processor frequencies, and they fall short under some thermally constrained scenarios. Drawing from recent advances in machine learning, we identify and take advantage of the key insight that not all tokens require full-depth inference to maintain high-quality generation. Motivated by this, we present PELM, a solution that augments traditional DVFS processor frequency tuning with two additional workload-specific knobs: 1) speculative decoding and 2) variable verification depth to expand the optimization space to multiple dimensions for more power efficient on-device LLM inference. In extensive evaluations across hardware platforms and datasets, PELM demonstrates superior performance compared to state-of-the-art power governing methods, with up to 23.1% speedup and 52.4% reduction in energy consumption, while maintaining comparable task performance. The source code is available at https://github.com/imec-nu/PELM.

0.64