CS Paper Radar

WINIC: An Automatic Cycle-Accurate Multi-Platform Microbenchmarking Tool for Instruction Performance Metrics

Tobias Rühl, Jan Laukemann, Georg Hager, Gerhard Wellein

cs.PF ·submitted 2026-11-10 12:20 UTC · v1
code/data

Abstract

Performance models allow developers to gain important insights into the interaction of their code with the hardware that executes it, and enable them to apply target-specific optimizations. To create in-core performance models for out-of-order processors, architecture-specific performance metrics like latency, throughput, and port binding of individual instructions are indispensable. For x86 microarchitectures, there already exist tools to automatically obtain those values; however, the extension of performance modeling tools like the Open Source Architecture Code Analyzer (OSACA) to support further architectures like RISC-V emphasizes the need for a multi-platform microbenchmarking tool. We present WINIC (What I Need Is Cycles), an automatic cross-platform microbenchmarking tool. Unlike previous tools, it can obtain throughput and operand-based latency metrics for most supported instructions of any 64-bit x86, AArch64, and RISC-V microarchitecture without the need for root permissions or extended system control levels. We cover the methodology of microbenchmarking and the approach of WINIC to collect cycle-accurate performance data for assembly instructions. We show that WINIC can be used to automatically obtain a vast number of accurate results and demonstrate for four target microarchitectures that our results match between 71% and 99% of the performance data of instructions obtained by similar tools and documentation. Furthermore, we investigate how operand-based metrics measured by WINIC can be used to improve the in-core performance predictions on selected micro-kernels, exemplarily for the Neoverse V2 microarchitecture.

Comment
12 pages, 7 figures, 3 tables, 10 listings

Closest papers

Nearest neighbours by meaning across the whole index.

Argus: Agentic, Reference-Calibrated, Tree-Guided, System-Software-Level Bottleneck Localization

cs.OS cs.AR ·2026-09-28 ·Vlad-Petru Nitu, Harsh Songara, Konstantinos Sgouras, Spiros Galanopoulos, Konstantinos Kanellopoulos +1 more
published similarity 0.71
Abstract · 5 pages, 7 figures. Extended version of the paper presented at the Architecture 2.0 Workshop at ISCA 2026

Operating system (OS) code can account for a substantial share of CPU execution time. First, as application logic is offloaded to heterogeneous accelerators (e.g., GPUs), the CPU increasingly acts as an orchestrator, spending cycles in driver calls, data movement, and synchronization rather than in application code. Second, workloads such as serverless functions frequently invoke OS services. At the same time, the OS is a complex codebase spanning many subsystems (e.g., memory management, networking), making it hard to localize the specific code path responsible for a slowdown. Existing profilers expose measurements that require interpretation(e.g., perf and Intel VTune) or can perturb short operations when extensively instrumented (e.g., ftrace). Diagnosing OS bottlenecks can therefore require repeated kernel instrumentation and manual interpretation. We introduce Argus, an agentic LLM-based profiler that produces instrumentation code and autonomously reasons over potential OS-level bottlenecks. Argus integrates two key mechanisms: (i) a calibration methodology that involves collecting a measurement from an idle system and using it as a reference point to discover potential bottlenecks, and (ii) a tree-based data structure that represents the different OS execution paths, improving the agent's bottleneck localization accuracy. Argus aims to identify a specific kernel code path rather than stop at a subsystem-level diagnosis. In two case studies, we employ Argus to autonomously discover bottlenecks present in the memory management subsystem caused by (i) a THP aggressor co-running with other applications, and (ii) applications that incur different types of page faults. Argus produces 19 times fewer incorrect deep-path diagnoses than the strongest evaluated LLM-based baseline, which lacks reference calibration, while preserving low time-to-diagnosis (approximately 31 s)

0.32

SPEC CPU2026 Under the Microscope: Programs, Phases, and Samples

cs.AR cs.PF ·2026-11-12 ·Peijun Li, Pei-Hung Lin, Tianming Cui, Caiwen Ding, Yang Katie Zhao +1 more
benchmark similarity 0.69
Abstract

SPEC CPU2026, the latest generation of the SPEC CPU benchmarks, is set to become the standard benchmark suite in computer architecture research, where new designs are often evaluated through sampled simulation with tools such as SimPoint. Existing characterizations of the suite focus on whole-run averages, which reveal neither how behavior changes within a run nor how well sampling captures it. We characterize CPU2026 on real hardware at three levels: programs, phases, and samples. At the program level, we report whole-run metrics and top-down bottleneck classes for all four suites and examine how they change with workload size, compiler version, and platform, across machines with three different instruction set architectures: x86, Arm, and RISC-V. Notably, changing only the compiler version can halve or nearly double the IPC of individual benchmarks. At the phase level, we track hardware counters every 10 ms over all 90 SPECrate inputs, identify recurring patterns of time-varying behavior, and divide the inputs into three groups: steady, multi-phase, and bursty. More than half of the inputs show time-varying behavior, and IPC within a single input can vary by more than an order of magnitude. At the sample level, we validate SimPoint against hardware measurements of every interval it can select. SimPoint is accurate on steady inputs, but its error reaches 43% on the others, and changing only the random seed spreads the error of one input over a range of 58 percentage points. Keeping SimPoint's clusters but choosing each representative by measured CPI brings every input within 2.6%, so the error comes from the choice of representatives, not from the clustering. We conclude with practical recommendations on sampling strategy for CPU2026, which give researchers a deeper understanding of the suite and show where additional simulation effort is needed to keep sampled results accurate.

0.18

The Unseen Delta: Characterizing the Compiler Optimization Landscape via Top-Down Differential Analysis

cs.SE ·2026-08-10 ·Zhibo Liu, Huaijin Wang, Shuai Wang
top venue similarity 0.69
Abstract · To appear at ISSTA 2026; figures fixed

Compiler optimizations are essential for achieving high performance in modern software. However, recent studies highlight the persistence of performance bugs, i.e., subtle defects where the compiler generates functionally correct but computationally inefficient code, leading to significant performance degradation. Existing detection and testing methods typically employ a bottom-up approach, focusing on specific low-level code properties and remaining confined to known optimization rules. Consequently, they struggle to quantify the holistic impact of identified issues and often overlook critical microarchitectural inefficiencies. We observe a key indicator of untapped potential: different compilers often produce binaries with significant performance differences for identical source code. However, the root causes of these discrepancies remain largely unexplored and difficult to pinpoint using current techniques. To bridge this gap, we introduce a top-down differential analysis methodology. This approach calibrates compiler optimization differences with fine-grained, hierarchical microarchitectural metrics, offering a comprehensive view of runtime behavior. Using a sampling-based approach, this method efficiently pinpoints the critical code snippets responsible for performance differences, enabling targeted root cause analysis. Our empirical evaluation uncovers substantial and often surprising performance differences between binaries generated by GCC and Clang. A categorization of root causes reveals systemic challenges in compiler optimizations. To quantitatively validate our findings and demonstrate practical impact, we developed a binary patching framework that fixes identified performance issues by transplanting superior code sequences from competing compilers. This work provides a novel lens for understanding and analyzing optimization defects.

0.40

LLTA: A Simplicity-Oriented Open-Source WCET Analyser

cs.PF ·2026-09-30 ·Nils Hölscher, Kay Heider, Jian-Jia Chen
theorycode/data similarity 0.68
Abstract

Deriving a safe upper bound on the worst-case execution time (WCET) of a real-time task is essential for hard real-time systems. Many WCET analysers exist, but they are either (i) closed source or (ii) do not provide a WCET for an existing microcontroller. Hence, it is difficult to obtain a trustworthy WCET bound for the binary that is flashed to an available embedded hardware platform and to observe whether the bound is safe. To solve this, we present LLTA, an open-source WCET analyser for Commercial-Off-The-Shelf (COTS) microcontrollers. LLTA is designed based on the following goals: it is simple (G1), runs on real COTS hardware (G2), reports results that are unsound due to programming or device constraints (G3), and is available open-source without licensing complications (G4). LLTA generates WCETs for the ESP32-C6 and the MSP430(FR5994), which are broadly available. This paper systematically lays out our design decisions for LLTA with those goals in mind. To demonstrate its ease of use, we propose scenarios where LLTA can be used for lab exercises of real-time systems.

0.46

Memory-Access Profiling for Early Multicore Interference Awareness in Embedded Systems

cs.PF ·2026-11-03 ·Jean-Baptiste Lefoul, Ahmad Shahnejat Bushehri, Alaric Chan Lock, Felipe Gohring de Magalhaes, Gabriela Nicolescu
benchmark similarity 0.66
Abstract

Applications executing on different multicore processors may interfere through shared resources. Such interference can introduce execution-time variability and complicate system integration, especially when applications that behave predictably in isolation are co-scheduled. This paper presents a profiling framework for early multicore interference awareness. The proposed approach captures source-level memory-operation traces from C applications with limited source-code modification and combines them with architectural parameters of the target platform. The framework maps memory accesses to cache sets and memory blocks, generates cache-occupancy profiles and heatmaps, and evaluates pairwise interference risks through a progressive three-stage analysis. First, a coarse-grained screening step classifies applications according to their non-stack memory-access behavior. Second, a static multicore cache-profiling step identifies cache sets whose combined footprint exceeds or approaches the hardware associativity. Third, a temporal footprint analysis tracks intervening unique blocks between consecutive accesses to the same memory block and computes Contention Potential Growth to identify execution windows vulnerable to cache eviction. The framework is evaluated using four benchmark pairings executed on a dual-core platform, with hardware performance counters measuring L2 cache misses used as validation references. The results show that the proposed analysis is consistent with the observed high- and low-risk cases and provides conservative warnings for near-capacity configurations. In particular, different applications pairs might produces up to 213.09% increase in L2 cache misses. These results demonstrate that memory-access profiling can provide practical and interpretable early indicators of cache-related multicore interference before exhaustive hardware testing.

0.24

HoliBench: A Cross-Platform Benchmarking and Deployment Toolkit for Foundation Models in CPS-IoT Applications

cs.DC cs.LGcs.PF ·2026-09-11 ·Inesh Chakrabarti, Zejun Xiong, Pragya Sharma, Mani Srivastava
code/data similarity 0.66
Abstract · 16 pages, 9 figures, 2 tables. Code: https://github.com/beesfleas/HoliBench

Foundation models, including large language models, vision-language models, and time-series foundation models, are increasingly deployed on embedded and edge platforms for CPS and IoT applications, where energy, latency, and memory are as critical as task accuracy. Existing benchmarking tools evaluate model capability in isolation, reporting accuracy assuming sufficient compute, while hardware profiling tools remain platform-specific and mutually incompatible. As a result, users lack a unified workflow for making deployment decisions across heterogeneous devices. We present HoliBench, a modular benchmarking and deployment toolkit that jointly characterizes accuracy, latency, and energy across platforms from single-board computers to GPU servers. Its platform abstraction layer calibrates cross-device measurement, and the toolkit supports multiple model modalities, inference engines, concurrencies, and existing evaluation harnesses. An interactive interface exposes constraint-aware configuration selection over a design space that is profiled once and reused across studies. Using HoliBench, we characterize 20 models across 7 device types, 3 quantization levels, 8 inference backends, and over 30 tasks, surfacing tradeoffs that existing tools miss: quantization reduces latency only on hardware with low-precision support, accuracy gains show diminishing returns relative to energy, and for autoregressive workloads, average inference power is approximately constant across output lengths. We further find that single-model profiles compose under sequential co-resident execution. In a multi-model CPS deployment, standalone profiles predict combined-pipeline latency and power within 1.2% and 2.5%, enabling deployment exploration without exhaustively profiling every pipeline configuration. We release HoliBench as open-source infrastructure for deployment-aware evaluation of foundation models.

0.36

BenchmarkAnything: Agent-Driven Construction of Simulator-Ready Microarchitecture Benchmarks

cs.AR ·2026-10-07 ·Xiangfeng Sun, Ceyu Xu, Chen Bai, Yuan Xie
benchmarkcode/data similarity 0.66
Abstract

The selection of benchmark workloads is of paramount importance in computer architecture, as it establishes the yardstick against which architectural innovations are measured and guided. Yet for decades, the SPEC benchmark suites, comprising merely tens of workloads, have been the de facto standard in academic architectural research, where they are frequently treated as a principal evaluation and optimization target. When a suite this small is relied upon so heavily, it risks architectural overfitting; as our research and prior studies demonstrate, an overly narrow focus can mislead design decisions by overvaluing certain innovations, producing cores that excel on SPEC benchmarks yet underperform on broader, realistic workloads. To mitigate this overfitting, adopting a large, comprehensive benchmark suite is the natural solution. However, the immense engineering effort required to strip software into the clean, interference-free binary executables demanded by simulators often makes this highly impractical. In this work, we demonstrate that AI agents provide an elegant solution to this challenge. Rather than manually curating yet another static benchmark suite, we introduce an agent-driven workflow capable of autonomously transforming arbitrary open-source repositories into simulator-ready executables. This automated approach makes workload collection highly scalable, allowing us to rapidly harvest hundreds of diverse applications from public repositories into our benchmark suite. Through a comparative analysis of our agent-generated suite against SPEC, we show that it not only achieves higher-fidelity performance assessments but also uncovers novel architectural insights that traditional, static suites fail to expose.

0.29

OpenMP Meta-Lowering: A Declarative Approach to Performance Portable Parallel Code Generation

cs.PL cs.DC ·2026-10-02 ·Luca Parigi, Giuseppe Tagliavini
published similarity 0.65
Abstract · 16 pages, 8 figures, 2 tables. Accepted at CGO 2027 (Salt Lake City, UT, USA)

The increasing diversity of parallel hardware challenges existing compilation flows. While OpenMP provides a portable abstraction for shared-memory parallelism, existing compilers tightly couple the frontend semantics with fixed lowering strategies. This design limits performance portability across different runtimes and architectures. In this paper, we present a modular approach to parallel code generation based on the Multi-Level Intermediate Representation (MLIR) framework. Our approach combines an OpenMP frontend with a domain-specific language (DSL) that specifies how OpenMP constructs are lowered for a target. We call this approach OpenMP meta-lowering, treating lowering as a programmable component rather than compiler-specific logic. This design provides explicit control over code outlining, data sharing, and runtime interfacing across diverse targets. We evaluate our approach on PolyBench/C-OMP across general-purpose and embedded multicore targets. Our results show that the proposed method matches the performance of state-of-the-art compiler toolchains with negligible code-size overhead (< 0.7%). Preserving OpenMP constructs until late lowering stages enables both general-purpose and OpenMP-specific MLIR optimizations. By decoupling lowering from compiler internals, our approach expresses the same OpenMP subset in 5,182 lines of code, ~32% fewer than the corresponding lowering code in Clang and ~76% fewer than in GCC, while supporting the pmsis runtime costs only a few specification lines, enabling rapid support for new runtimes without modifying the compiler.

0.38