CS Paper Radar

Batch Before You Time: Decision-Scoped Proxy Execution for Timing-Aware Logic Rewriting

Pujun Su

cs.AR ·submitted 2026-09-02 11:39 UTC · v1

Abstract

Standard Delay Format (SDF)-annotated switching simulation distinguishes delay-dependent activity among functionally equivalent rewrites, but evaluating every candidate repeats timing, compilation, and replay. A zero-delay proxy can remove timed evaluations, but generating that proxy candidate by candidate can cost more than the timed work it saves. We present Batch Before You Time (BBYT), which compiles all candidates of one rewrite decision into one scoped zero-delay image and either commits a well-separated proxy winner or invokes the unchanged timed chain. Across 12 counterbalanced holdout sequences, BBYT reduces complete candidate-selection time by 18.05% on average; the design-level reductions are 5.84% and 30.25%, with both confidence intervals above zero. On a counterbalanced 8,192-transition C6288 workload, BBYT is 9.24% faster than the same gate executed with candidate-wise proxy launches. In the five-workload corpus, BBYT removes 32.52% of timed candidate evaluations and matches exhaustive timed selection on all 250 evaluated decisions. For on-demand timing-aware rewrite selection, proxy execution and fidelity continuation should use the same decision scope.

Comment
6 pages, 3 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Tool Waiting and Re-arrival in Compile-Time-Static LLM Serving: Cost Mechanisms and Configuration Selection

cs.PF ·2026-09-28 ·Dongkyeom Jang, In-Nea Wang, Junho Jeong
similarity 0.69
Abstract · 24 pages, including 7 pages of supplementary material

In agentic LLM services, a session calls an external tool, waits for it, and re-arrives to continue inference. Statically compiled NPU serving can fix the batch bucket set, the maximum batch size, and the number of KV cache slots at compile time. We define such an environment as a compile-time-static serving substrate and analyze the execution-time cost that tool waiting and re-arrival incur in it. On a single LLM instance, we run synthetic workloads following a measured tool waiting time distribution and compare, on the same inputs, a baseline configuration with settings {1, 2, 4, 8}, 8, and 8 against configurations that change some of them. Because re-arrival times differ across configurations, we build a simulator that replays request processing in time order, select the candidate with the lowest predicted cost among 2,077 configurations, and validate it on new inputs. We identify three mechanisms: discrete batch alignment, KV cache survival, and prefill interference. At a concurrency of 6, absent from the bucket set, tool waiting lowered the padding ratio (0.235 to 0.120) yet increased decode execution time 1.51-fold, so padding alone did not indicate cost. On new inputs at a concurrency of 8, where the baseline reused KV in 9 of 24 re-arrivals, enlarging the maximum batch size alone cut execution cost by 8.25%, and the selected configuration, which also adjusted the bucket set, by 9.72%. Where 17 of 18 re-arrivals were already reused, the effect was 0.59%. Compile-time configurations should thus be selected by diagnosing KV reuse loss and the resulting change in execution.

0.27

TicTacBench: Benchmarking Timing Closure Capabilities of Coding Agents

cs.AI ·2026-09-20 ·Bowei Wang, Zhigang Fang, Zhijie Yang, Renzhi Chen, Shanshan Li +1 more
publishedbenchmark similarity 0.63
Abstract · Accepted at ICCD26

Recent advances in large language models (LLMs) have led to the emergence of coding agents capable of performing complex engineering tasks, including register-transfer level (RTL) design and optimization. Existing RTL benchmarks mainly evaluate functional correctness and performance, power, and area (PPA) of the generated RTL designs, leaving agents' ability for \emph{timing closure} under-evaluated. We propose TicTacBench, a benchmark specifically designed to evaluate coding agents' capabilities for RTL-level timing closure under post-place-and-route (post-PnR) evaluation. TicTacBench contains 30 diverse tasks, each provided with a suboptimal RTL design, realistic timing constraints, functional equivalence verification, and timing reports. With over 300 runs of coding agents driven by 8 frontier LLMs, we find that even the best agent can only close 53.3\% of tasks with 7.18\% area-delay product (ADP) degradation and 8.83\% energy-delay-squared product (EDDP) improvement on average. We identify common failure categories that explain why agents fail to close timing. Then we propose TicTacSkill, a new method that guides agents to follow standard timing-closure procedures and improves the Timing Closure Rate by 9\%. These results suggest that while coding agents have made significant progress in RTL design, their timing-closure capability still has substantial room for improvement.

0.30

CAGE: Regime-Aware Algorithm Engineering for Coverage Optimization

cs.AR ·2026-08-28 ·Amirreza Khorasanian
hot direction similarity 0.61
Abstract · 21 pages, 9 figures. Submitted to Algorithmica. Research artifact: https://doi.org/10.5281/zenodo.22114770

Coverage-selection policies that optimize the same monotone objective can occupy different quality-runtime frontiers as workload scale, budget, repetition horizon, and execution substrate change. CAGE is a regime-aware algorithm-engineering framework that holds coverage semantics fixed through a shared bitset execution substrate and studies how the nondominated strategy set changes under controlled operating conditions. We separate intrinsic workload descriptors from measured strategy response, represent each condition by its empirical Pareto set, and identify regime transitions from changes in Pareto membership or statistically supported crossovers. Across Defects4J, Epinions, a BWSN-inspired time-to-detection workload, repeated selection, and an embedded RV32 realization, we observe distinct transitions. At a 5% budget, one-exchange refinement adds 27 Chart lines at 11.3x baseline median runtime, whereas deeper search adds five Time lines at roughly 715x. On Math, CP-SAT confirms full-universe optimality at 20% and 40% and residual lower-budget headroom. Persistent preprocessing reuse first achieves confidence-interval-supported speedup at horizon 20 and reaches 1.442x at horizon 100. ModelSim shows a separate bitset-width crossover. The results demonstrate that strategy nondominance is workload- and condition-dependent, supporting regime-aware algorithm engineering rather than a universal policy ranking.

0.45

VelServe: Taming Decode Jitters in LLM Inference with Tail-Calibrated Scheduling

cs.AI ·2026-11-13 ·Ishaan Datta, Alind Khare, Koushik Kumar Dey, Ratnesh Jamidar, Anand Kumar Singh
similarity 0.61
Abstract

Interactive LLM services must meet latency service-level objectives (SLOs) for the time to first token (TTFT) and the time between tokens (TBT), as streaming and soft real-time applications require predictable token delivery. We identify decode jitters in both aggregated and disaggregated serving. A jitter delays every request in the batch, and long responses span hundreds of steps, so even rare overruns cause many SLO violations. In aggregated serving, SLO-aware schedulers set each step's prefill budget until its predicted latency just meets the TBT target, leaving no room for mispredictions or host stalls. Disaggregation meets average TBT targets at a cost in capacity, yet the steps that admit new requests still run late. We present \sysname, a scheduler that calibrates how much prefill work each decode step carries using the measured tail of its own iteration latency. The SLO determines how rare a late step must be, and \sysname leaves enough headroom in each iteration to prevent overruns and meet the target decode latency. This headroom slows prefill, so \sysname serves the shortest waiting prompt first. When TTFT deadlines grow with prompt length, the long prompts it delays have the most time to spare. Under disaggregation, it spaces admissions so that no request waits on more slow admission steps than its P99 target allows. On three attention families and varying workloads, \sysname sustains 2.4--2.9$\times$ the SLO goodput of the strongest baseline on Qwen3-8B and DeepSeek-V4-Flash, and at least 4.4$\times$ on DeepSeek-V2-Lite. With prefill and decode disaggregated on eight GPUs, \sysname meets the SLO for more than twice as many requests as state-of-the-art baselines.

0.19

Twice as Fast, Until It Isn't: Quantized Serving of Mid-Size Open LLMs

cs.AI ·2026-11-03 ·Smit Kadvani, Kriti Faujdar, Spurthi Tallam
benchmark similarity 0.61
Abstract

A team serving a 9B-20B open-weight language model on vLLM or SGLang must choose a quantization recipe, and needs two numbers: the speedup over BF16 and the accuracy given up. We assemble every public measurement of these numbers that we could verify and find that the record supplies them only in pieces. Only two checkpoints, both INT4 builds of Gemma 3 12B, have both a vLLM timing and an accuracy score, and that score covers a single MMLU subject, one that the AWQ build was calibrated on. The batch-1 timings predate current engine releases, AMD and Intel Gaudi have no timing in this band at all, and FP4 has none against BF16. The timings that do exist follow one rule. Decoding a small batch is limited by memory bandwidth, so 4-bit weights roughly double speed (2.12x for Qwen2.5-14B on an A100), but the gain vanishes once a large batch makes the GPU compute-bound (1.02x for Gemma 3 12B at batch size 256), where FP8 is the recipe that can still help. Many published accuracy deltas, including every HumanEval delta, lie within benchmark sampling noise. We close with a seven-item reporting checklist for future measurements.

0.29

RadixFlow: Filling Execution Bubbles in Prefix-Cached Hybrid LLM Serving

cs.AI ·2026-10-13 ·Huihao Jing, Wenbin Hu, Shaojin Chen, Haochen Shi, Haoyu Huang +4 more
similarity 0.60
Abstract

Prefix reuse improves LLM inference by sharing the KV cache of common prompt prefixes. This helps only if the scheduler can execute a matched branch in the current worker round. Schedulers assume that a matched prefix is ready, which holds for full attention but fails for hybrid LLMs. There, a branch (the consumer) may need recurrent state that another branch (the producer) is still computing in the same round. The native scheduler stops its waiting-queue scan at this blocked consumer, and the resulting head-of-line blocking creates \emph{execution bubbles}: idle batch slots that independent work could fill. We introduce RadixFlow, which decouples prefix matching from state readiness. Its state-ready traversal turns a global scan barrier into a local wait: only the consumer is blocked until the next round, while feasible independent branches keep filling the batch under the native state check. On five hybrid models, RadixFlow lowers mean latency by 3\% to 24\% and worker rounds by 11\% to 44\% at almost unchanged reuse. On the larger Qwen3.5 models, ablations show that most of the gain comes from traversal, which adds little on a full-attention control. Heavier rounds limit how much of this saving reaches latency.

0.32

State-Aware Activation Recomputation with Replay Contracts and Selective Preservation

cs.AI ·2026-10-31 ·Annan Zhang, Shengjie Song, Wenshuai Yao, Lihao Zhang, Tianmiao
similarity 0.60
Abstract

Activation recomputation saves accelerator memory by replaying forward segments during the backward pass, but replay is only correct if it sees the state the original forward pass saw. Mutable buffers, aliased bytes and random-generator state can change in between, so re-executing unchanged code can differentiate a different computation. We make this state explicit. A versioned replay contract specifies what replay must restore, and a joint planner chooses, per segment, between activation retention, whole-buffer snapshots and selective-region preservation, charging each option its measured memory and recovery cost. Independent derivative and byte-level checks validate the contract, and a Torch/JAX cohort retains all 5,900 execution records, including matches, rejections and divergences. On frozen sparse-write H100 workloads, each measured over fifty fresh processes, joint planning reduces paired step time by 18.16% against a correctly costed eager/whole planner at a 44 GiB budget (approximate 97.5% ratio interval $[.8172,.8195]$). At 32 GiB, where the eager/whole planner has no feasible static policy, joint planning is 4.98% faster than fixed regions at 29.94 rather than 26.20 GiB. A complete CIFAR-100 dense-queue application runs seven methods for 100 encoder and 25 probe epochs at six training seeds, with byte-identical test scores within each seed. There, joint planning uses 22.41% more allocated memory than whole snapshots, with a process-time geometric-mean ratio of 1.0080 (seed range $[.9845,1.0191]$). Together, the results map where selective preservation pays off: sparse writes with large activations benefit, while in the dense-queue workload whole snapshots use less allocated memory.

0.26

Control-Flow Profiling for Workload Representativeness

cs.PF cs.AR ·2026-11-12 ·Mahesh Madhav
similarity 0.60
Abstract · 11 pages, 7 figures, 5 tables

Workload characterization in performance engineering often replaces expensive target executions with lower-cost proxies, including reduced datasets, miniapps, and training inputs. Such substitutions require empirical verification that the proxy faithfully exercises the target's dynamic execution behavior. This paper presents an address-keyed control-flow profiling methodology and tool suite designed to evaluate workload representativeness directly from compiled, unmodified binaries. We pair time-ordered basic-block vector (BBV) recurrence analysis with exp-branchprof, an experimental Valgrind tool that captures five orthogonal structural profile families spanning branch direction bias, four-bit global history contexts, dynamic indirect targets, function call transitions, and intraprocedural CFG edges. Our method compares weighted event distributions from two executions of the same binary. It reports support coverage, reference event-mass coverage, distribution distance, entropy-derived quantities, and phase structure, rather than claiming that a single similarity score certifies a proxy. We validate the tool with exact-oracle microbenchmarks across AArch64 and x86-64, covering conditional and inverted exits, calls, returns, direct and indirect jumps, and forked processes. We then apply the workflow to SPEC CPU2017 and CPU2026 train/reference pairs and seven refrate/refspeed pairs. The corpus contains both close matches and cases where a shorter execution reaches much of the same code but misses substantial target, callgraph, CFG, or BBV event mass. The accompanying artifact supports the same analysis for proprietary workload/proxy pairs without requiring raw PC-level profiles to leave the collection environment.

0.29