CS Paper Radar

Symbolic Basic Block Profiling for Machine Learning Kernels

Jingyu Qiu, Rongcui Dong, Sreepathi Pai

cs.PL ·submitted 2026-08-20 22:56 UTC · v1
published

Abstract

Current basic block profiling techniques obtain the count of executions of each basic block in a program using dynamic instrumentation. These profiling counters create runtime overheads and also require the execution of the program, which, for large input sizes, can take substantial time. We propose symbolic program profiling that generates symbolic formulae for a basic block's count with inputs as the independent variables. Our technique is limited in applicability to a certain class of programs, namely machine learning (ML) kernels. We implement our technique in the LLVM compiler and evaluate it on 78 ML operators from 50 different ML models. These operators are generated by TVM, a machine learning compiler. Our symbolic profiles deliver exactly the same results as dynamic instrumentation for 73 out of 78 kernels with a median speedup of 15093x.

Comment
Accepted to OOPSLA 2026
DOI
10.1145/3839483

Closest papers

Nearest neighbours by meaning across the whole index.

Less is more: learning what to profile for GPU kernel optimization

cs.LG cs.AI ·2026-10-27 ·Yihan Wang, Peiyu Liu, Runyu Chen, Wei Xu
code/datahot direction similarity 0.66
Abstract

Large language models have recently shown strong potential for automatically optimizing GPU kernels, yet their effectiveness depends critically on the hardware information available to the kernel optimization model. Existing systems typically rely on predefined profiling strategies, leaving what to profile largely outside the optimization process. We show that effective profiling is not simply about collecting more information: increasing the number of profiling metrics does not monotonically improve kernel optimization, while the most useful profiling information varies across kernels. Our theoretical analysis further shows that, under such heterogeneous profiling utility, kernel-conditioned selection can outperform any single fixed profiling configuration. Motivated by this observation, we propose AutoProfiler, a feedback-driven framework for learning a deterministic profiling tree from optimization feedback. The tree is iteratively refined through REWRITE and SPLIT operations, with an LLM generating candidate modifications and empirical evaluation determining their acceptance. Once constructed, the tree quickly gathers the required profiling information for any kernel, allowing the LLM rewriter to be invoked only once. Experiments on KernelBench show that our approach improves kernel optimization over fixed profiling configurations under constrained profiling budgets. These results suggest that hardware profiling should evolve from static metric collection toward profiling reasoning, making profiling itself a new dimension for improving kernel optimization. The codes are available at https://github.com/Satissss/AutoProfiler.

0.31

HPC-Bench: A Comprehensive Benchmark for High Performance Computing Codes

cs.PF ·2026-11-02 ·Bowen Cui, Junyu Yin, Tejas Ramesh, Leo Lim, Oscar Hernandez +1 more
publishedbenchmarkcode/data similarity 0.62
Abstract · Accepted at NeurIPS 2026 Evaluations & Datasets Track (poster). 31 pages, 13 figures, 19 tables

Large Language Models (LLMs) have shown strong capabilities in code generation and reasoning, motivating extensive benchmarks for evaluating functional correctness. However, performance optimization in High-Performance Computing (HPC) poses fundamentally different challenges that are not captured by existing evaluations: HPC performance depends on low-level code, intricate memory access patterns, and computational kernels embedded within sophisticated execution contexts. We introduce HPC-Bench, a performance-grounded benchmark for evaluating LLMs as optimizers of computational kernels in realistic HPC workloads, including 114 workloads across 16 HPC computational motifs. HPC-Bench provides a fully automated, end-to-end evaluation pipeline that combines program-level correctness verification with two correctness-gated metrics, Fast@k and Speedup@k, to characterize reliability and speedup under a fixed sampling budget. Our evaluation of three representative LLMs across serial CPU, OpenMP, and CUDA settings shows that stricter speedup requirements substantially reduce the probability of finding a correct and fast optimization. HPC-Bench is available at https://github.com/Deep-Learning-Profiling-Tools/HPC-Bench.

0.48

Argus: Agentic, Reference-Calibrated, Tree-Guided, System-Software-Level Bottleneck Localization

cs.OS cs.AR ·2026-09-28 ·Vlad-Petru Nitu, Harsh Songara, Konstantinos Sgouras, Spiros Galanopoulos, Konstantinos Kanellopoulos +1 more
published similarity 0.62
Abstract · 5 pages, 7 figures. Extended version of the paper presented at the Architecture 2.0 Workshop at ISCA 2026

Operating system (OS) code can account for a substantial share of CPU execution time. First, as application logic is offloaded to heterogeneous accelerators (e.g., GPUs), the CPU increasingly acts as an orchestrator, spending cycles in driver calls, data movement, and synchronization rather than in application code. Second, workloads such as serverless functions frequently invoke OS services. At the same time, the OS is a complex codebase spanning many subsystems (e.g., memory management, networking), making it hard to localize the specific code path responsible for a slowdown. Existing profilers expose measurements that require interpretation(e.g., perf and Intel VTune) or can perturb short operations when extensively instrumented (e.g., ftrace). Diagnosing OS bottlenecks can therefore require repeated kernel instrumentation and manual interpretation. We introduce Argus, an agentic LLM-based profiler that produces instrumentation code and autonomously reasons over potential OS-level bottlenecks. Argus integrates two key mechanisms: (i) a calibration methodology that involves collecting a measurement from an idle system and using it as a reference point to discover potential bottlenecks, and (ii) a tree-based data structure that represents the different OS execution paths, improving the agent's bottleneck localization accuracy. Argus aims to identify a specific kernel code path rather than stop at a subsystem-level diagnosis. In two case studies, we employ Argus to autonomously discover bottlenecks present in the memory management subsystem caused by (i) a THP aggressor co-running with other applications, and (ii) applications that incur different types of page faults. Argus produces 19 times fewer incorrect deep-path diagnoses than the strongest evaluated LLM-based baseline, which lacks reference calibration, while preserving low time-to-diagnosis (approximately 31 s)

0.32

Control-Flow Profiling for Workload Representativeness

cs.PF cs.AR ·2026-11-12 ·Mahesh Madhav
similarity 0.61
Abstract · 11 pages, 7 figures, 5 tables

Workload characterization in performance engineering often replaces expensive target executions with lower-cost proxies, including reduced datasets, miniapps, and training inputs. Such substitutions require empirical verification that the proxy faithfully exercises the target's dynamic execution behavior. This paper presents an address-keyed control-flow profiling methodology and tool suite designed to evaluate workload representativeness directly from compiled, unmodified binaries. We pair time-ordered basic-block vector (BBV) recurrence analysis with exp-branchprof, an experimental Valgrind tool that captures five orthogonal structural profile families spanning branch direction bias, four-bit global history contexts, dynamic indirect targets, function call transitions, and intraprocedural CFG edges. Our method compares weighted event distributions from two executions of the same binary. It reports support coverage, reference event-mass coverage, distribution distance, entropy-derived quantities, and phase structure, rather than claiming that a single similarity score certifies a proxy. We validate the tool with exact-oracle microbenchmarks across AArch64 and x86-64, covering conditional and inverted exits, calls, returns, direct and indirect jumps, and forked processes. We then apply the workflow to SPEC CPU2017 and CPU2026 train/reference pairs and seven refrate/refspeed pairs. The corpus contains both close matches and cases where a shorter execution reaches much of the same code but misses substantial target, callgraph, CFG, or BBV event mass. The accompanying artifact supports the same analysis for proprietary workload/proxy pairs without requiring raw PC-level profiles to leave the collection environment.

0.26

MLIR-MRacle: Memory-Model-Aware Metamorphic Fuzzing of Concurrent MLIR

cs.SE ·2026-11-18 ·Aaron Gabayoyo, Aidan Dakhama, Ajitha Rajan
similarity 0.61
Abstract · 4 pages, 1 figure

Multi-level compilers built on MLIR lower concurrent programs through many stages towards targets with potentially drastically different memory models, where an optimisation which appears reasonable, can silently change program behaviour. Testing these lowerings is hard: a concurrent program often has no single expected outcome, and existing MLIR fuzzers often assume determinism. We present MLIR-MRacle, a memory-model-aware metamorphic fuzzer for concurrent MLIR programs. MLIR-MRacle applies metamorphic relations gated by the target memory model, so that both the original and transformed programs remain well defined - executing both repeatedly, and comparing their outcome sets or distributions withone of two statistical oracles. A coverage guided campaign driver built on libafl is used to perform continuous fuzzing. MRs, memory models, oracles, and execution backends all provide extension points for future researchers and compiler engineers to build off this package. In a case study on Armv8 hardware, MLIR-MRacle ran over 145k trials across four configurations, reaching up to 93.6k compiler coverage edges, and found a bug in the OpenMP dialect verifier that has since been fixed upstream. Video: https://youtu.be/lKtGk0H97Ts?si=dWLUHl2CJ24ep6JY

0.35

SysPilot: Cross-Library Performance Autotuning via Library Performance Contracts

cs.PF cs.AI ·2026-11-04 ·Yineng Yan, Haowei Chen, Christopher J. Rossbach
benchmarkcode/data similarity 0.60
Abstract · SysPilot is open source at https://github.com/yinengy/SysPilot

Modern scientific and machine-learning applications compose many independently developed libraries. When these libraries run together, their independently managed runtimes can oversubscribe shared resources: using the same CPU cores, fighting over GPUs, or contending for memory bandwidth. Avoiding such contention requires careful configuration across library boundaries. Existing autotuners do not directly investigate what resources are shared between libraries, which runtime signals correlate with interference. As a result, they are inefficient for finding optimal configurations to mitigate contention. This paper presents SysPilot, a cross-library performance autotuning system based on Library Performance Contract (LPC). A Library Performance Contract captures a library's resource demands, assumptions, controls, and observability signals in LPCL, a specification language for describing library resource requirements. SysPilot synthesizes contracts from documentation and code artifacts, composes contracts from the libraries used by an application, derives resource-conflict hypotheses, profiles the application to validate those hypotheses, and selects candidate configurations. This design makes the Library Performance Contract a reusable unit of performance knowledge: a contract for a library can be reused across many applications that depend on that library. We evaluate SysPilot on a benchmark suite of cross-library contention scenarios. The results show that SysPilot matches the performance of autotuners that rely on application-specific manual information while retaining cross-application reuse. Compared with general-purpose autotuners such as OpenTuner and SMAC3, SysPilot improves benchmark performance metrics by up to an additional 38% over the best configuration found by these autotuners, while using 3x fewer runs.

0.44

Cleave: Scaling Tensor Program Optimization via Decoupled Algebraic Search and Operator Scheduling

cs.PL cs.AI ·2026-10-06 ·David Pissarra, Jinkun Lin, Haitian Jiang, Aurojit Panda, Jinyang Li
code/data similarity 0.60
Abstract

Optimized kernels such as FlashAttention and FlashDecoding are crucial for accelerating today's large models. Most of them are handwritten by experts because existing ML compilers cannot match their efficiency. Producing such kernels requires fusing computations with multiple reductions, which requires both algebraic transformation of the computation graph and operator scheduling of the transformed graph. Unfortunately, searching the two jointly yields a space too large to navigate. We propose Cleave, an ML compiler built on symbolic decoupling: Cleave discovers transformations by performing superoptimization on a graph with symbolic shapes, and then schedules each resulting graph on concrete shapes. Representing shapes as symbols makes equivalence checking cheap and lets a new Split operator, with a symbolic split count, parallelize along a reduction dimension. Cleave's scheduler fuses graphs with multiple reductions through iterative tiling and horizontal fusion. Evaluation on common LLM subgraphs shows that Cleave generates kernels up to 2.8x faster than the best baseline (1.6x on average) and reduces compilation time by 5.9x on average compared to Mirage. For dynamic workloads captured from production serving traces, Cleave compiles each operator once and achieves geometric mean speedups of 1.4x and 1.7x over FlashInfer's handwritten FA2 and FA3 backends. Cleave's code is available at: https://github.com/nyu-systems/cleave

0.26

DLCB: Ahead-of-Time Compilation for Dynamic Deep Learning

cs.PL ·2026-08-19 ·Alexander Collins, Bin Fan, Evghenii Gaburov, William Brandon, Sean Lee +2 more
similarity 0.60
Abstract

Deep learning workloads are increasingly deployed in settings where tensor shapes are not fully known at compile time. Batch sizes vary across requests, sequence lengths differ between inputs, and model architectures admit a range of spatial resolutions. This dynamism creates a fundamental tension: ahead-of-time compiled GPU kernels deliver peak performance but traditionally require fully static tensor shapes, while just-in-time compilation supports dynamic shapes at the cost of runtime compilation overhead. We present DLCB (Deep Learning Compiler Backend), a deep learning compiler that resolves this tension through a unified three-tier compilation strategy. For programs with fully static tensor shapes, DLCB generates static CUDA C++ kernels ahead of time. For programs whose tensor rank is statically known but whose dimension sizes are dynamic, DLCB generates AOT kernels parameterized by those dimensions, compiling once and executing across a range of shapes. For programs with tensors of unknown rank, DLCB falls back to JIT compilation at runtime. For AOT compilation with dynamic shape tensors, our approach builds a system of shape constraints from the input program, and uses a solver to reduce these to either a) fixed constraints that produce hard-coded constants in the generated kernel; b) unresolved constraints which become kernel parameters passed at launch time, along with interpreted host code to check that the constraint holds for a given set of inputs and to compute the values to pass. Our compiler and runtime are embedded in PyTorch, and use an automatic shape generalization pass to enable AOT compilation with dynamic shapes for supported subsets of PyTorch programs

0.14