CS Paper Radar

How Efficient Is Your Agent? Online Predictive Bounds under Budget Constraints

Roy Maor Lotan, Shai Feldman, Yaniv Romano

cs.LG ·submitted 2026-11-08 10:05 UTC · v1
theory

Abstract

We evaluate AI agents in an online setting where tasks arrive sequentially and, before running each task, we construct an upper predictive bound on its \emph{time-to-success}---the number of interactions until the task is completed successfully. Under a limited budget, some runs are stopped early, so their time-to-success is unobserved (right-censored), and the correctness of the bound may be unknown. We introduce \emph{\textbf{PR}edictive Bounds for \textbf{O}nline \textbf{B}udget \textbf{E}fficiency} (\texttt{PROBE}), a method for constructing calibrated upper predictive bounds while adaptively allocating a limited budget across agent tasks to reduce the anticipated coverage deficit. Our method calibrates the bounds online using inverse-probability-weighted feedback together with a correction designed to minimize coverage deficit while remaining unbiased. We prove that \texttt{PROBE} provides a coverage guarantee for any finite time horizon while satisfying the budget constraint. Experiments on software engineering, retrieval-augmented generation, and instruction following show that \texttt{PROBE} attains a coverage rate closer to the target level with tighter predictive bounds than a calibrated baseline.

Closest papers

Nearest neighbours by meaning across the whole index.

Dynamic Budget Allocation for LLM Evaluation under Hard Resource Constraints

cs.LG ·2026-10-05 ·Shai Feldman, Yaniv Romano
benchmarktheory similarity 0.72
Abstract

We evaluate large language models (LLMs) in multi-turn interactions through their time-to-event: the number of interaction steps required to produce an event of interest, such as a successful jailbreak or agentic task completion. Under limited compute, interactions may be terminated before the event occurs, so that event times are only partially observed (censored). Existing allocation methods for calibrating time-to-event bounds satisfy the budget only in expectation and can exceed the available budget on a particular evaluation run. Enforcing a hard constraint is particularly challenging as the cost of a trajectory is initially unknown. We introduce Hard-budget Allocation with Reflow for Predictive calibration (HARP), a budget allocation that satisfies hard resource constraints and adaptively reallocates unused budget. We show how to use HARP to construct lower predictive bounds (LPBs) on the time-to-event and to estimate evaluation metrics such as the jailbreak rate on a fixed benchmark. Although HARP induces dependence in acquisition decisions across different trajectories, we prove that HARP never exceeds the target budget, that its LPBs have finite-sample coverage guarantees, and that its metric estimates are unbiased. Experiments on agentic task success, LLM jailbreaks, toxic content generation, and RAG hallucinations show that HARP achieves coverage close to the nominal level with low variance, while never exceeding the given budget.

0.35

Chance-constrained selection of sequential intervention strategies from counterfactual estimates

cs.LG stat.MEstat.ML ·2026-08-13 ·Minkyoung Kim, Beakcheol Jang
code/data similarity 0.64
Abstract · 46 pages, 10 figures, 20 tables. Includes supplementary material as an appendix

Many operational decisions are sequences of interventions under a cumulative resource limit, such as a maintenance schedule within a crew-hour budget. Choosing among them calls for the outcome and the cumulative cost each would produce, counterfactual quantities identified from observational data. Two strategies with the same expected cost can exceed the budget at very different rates, so constraining the mean does not bound how often an overrun occurs. Prior two-step architectures, recently extended to continuous doses, constrain the mean cost rather than its tail and allocate at a single decision point. Methods that do bound a cost tail take its distribution from a specified model rather than identifying it from data. We present a predict-then-optimize framework. In the prediction step, any estimator returning an outcome value and a cost distribution supplies what the decision rule consumes, so the predictor is interchangeable. In the optimization step, a chance-constrained selection over a finite candidate set bounds the probability that the cumulative cost exceeds the budget. That tail does not decompose across stages, so each strategy is scored whole. Sweeping the tolerated violation probability traces a safety-utility frontier, and distribution-free finite-sample bounds cover violation and outcome shortfall. Four of five environments, spanning clinical treatment and equipment maintenance, supply exact counterfactual ground truth; the fifth carries real outcomes from a digital-health micro-randomized trial. Across them, the rule holds the budget where a point-estimate rule overruns it, at an outcome cost the frontier makes explicit. All code is available at https://github.com/mfriendly/counterfactual-chance-selection

0.30

Intelligence Under Time Constraints: Rethinking Test-Time Compute

cs.CL ·2026-09-14 ·Xiaotian Zhang
similarity 0.64
Abstract · Position paper. 10 pages, 1 figure, 3 tables

Intelligence under time constraints requires deciding not only how much to compute, but when computation is worth starting. We study this problem in streaming interactions, where evidence arrives incrementally and may be revised. Early computation has more time to finish but rests on incomplete evidence; waiting improves information while shrinking computational slack. We call this the information-slack dilemma. We take the evidence-dependent computational job as the unit of analysis: when to start it, what supports its result, and when that result can be committed. Advance computation is valuable only insofar as its benefits survive the costs of verification, invalidation, and recovery. This applies to grounded incremental processing and reusable preparation as well as future-dependent speculation. We propose a research agenda on computation under evolving evidence, prioritizing selective recovery under controlled evidence revisions. Evaluation should separate earlier-execution effects, deployment value against a full-input alternative, and the added value of predictive policies, while accounting for shared-resource costs. The objective is not maximal advance computation, but more trustworthy, on-time responses within a declared resource envelope.

0.19

Learning Robot Metareasoning: Adaptive Computation Budgets for Real-Time Control

cs.RO ·2026-10-16 ·Lorenzo Magnino, Antonio Marino, Michael Amir, Daigo Shishika, Amanda Prorok
similarity 0.63
Abstract

Planning while moving faces an historical trade-off between deliberation and reactivity. A planner running concurrently with execution needs time to compute, so the robot must commit to an open-loop action chunk for that duration: a longer commitment buys a better plan at the cost of feedback frequency. Existing controllers fix this action commitment, and consequently the computation budget, before deployment. However, the optimal computation time is highly state-dependent (due to factors such as unmodeled dynamics, safety constraints, and marginal optimisation gains), i.e. dynamic, therefore offline static tuning risks leaving systems blind during critical maneuvers or unable to find global effective solutions. We cast the identification of the optimal computation budget as a semi-Markov decision process (SMDP) in which a single decision defines both the robot action commitments and the time allocated to the solver. To solve this, we train a reinforcement-learning policy conditioned on the current state, the executing plan, and inexpensive planner statistics. Across ground-robot navigation, agile quadrotor flight, and contact-rich manipulation, the learned policy changes its budget with the demands of the scenario, discovering efficient reasoning patterns that outperform fixed-budget baselines. https://lorenzomagnino.github.io/learning-robot-metareasoning-review/

0.23

RBGA: Residual-Budget Guide Arbitration for Offline-to-Online Safe Reinforcement Learning in Dexterous Manipulation

cs.LG ·2026-10-24 ·Zongyou Yu, Ao Zhou, Xiongbo Shi, Jiayi Guan, Shihang Du +5 more
code/data similarity 0.63
Abstract

Dexterous manipulation requires robots to leverage existing datasets for rapid skill acquisition while maintaining safety during real-world deployment to prevent damage to the robot or its surroundings. Offline-to-online safe reinforcement learning (O2O safe RL) provides a promising solution for this problem by leveraging prior experience for online improvement under safety constraints, thereby mitigating the sample inefficiency and exploration risk of learning safe policies from scratch. However, existing O2O safe RL approaches do not explicitly account for the safety budget already consumed when making decisions within an episode. Excessive cost incurred early can therefore leave little room for subsequent exploration and policy improvement. To address this limitation, we propose \textbf{R}esidual-\textbf{B}udget \textbf{G}uide \textbf{A}rbitration (RBGA), an algorithm that explicitly tracks the remaining episode safety budget and uses it to adaptively select between offline guidance and the evolving online policy. At each step, RBGA compares the cost-to-go estimates of the guide and online actions against the remaining budget, enabling step-wise action arbitration throughout the episode. We provide theoretical analysis characterizing the safety implications of residual-budget-based selection and its effect on guide-assisted exploration. Extensive experiments demonstrate that RBGA enables effective online policy improvement while maintaining strong safety performance. Our code is available at this \href{https://github.com/sii-pilab}{link}.

0.21

AFA-BANDIT: Provably Near-Optimal Online Multi-Feature Classification Under Budget Constraints

cs.LG ·2026-10-06 ·AbdAlRahman Odeh, Teng-Hui Huang, Hesham El Gamal
theory similarity 0.63
Abstract · 10 pages, 3 figures

Active Feature Acquisition (AFA) is a classification problem in which an agent decides which costly features to acquire before predicting each sample's label. Unlike batch AFA, which trains a fixed policy and classifier offline on fully observed data, online AFA updates its predictor from revealed labels as samples arrive. Existing online methods either use deep reinforcement learning (RL) without performance guarantees or maximize cost-adjusted reward rather than enforce a global budget. We formulate online AFA as a combinatorial Bandits with Knapsacks (BwK) problem that couples acquisition and prediction. Unlike prior bandit-based AFA and classical BwK, our setting has combinatorial complexity, evolving rewards, a global budget, and structured side information. We obtain an improved regret upper bound over standard BwK bounds in this framework, leveraging a cardinality-aware confidence bound and the subset update structure. To avoid an exponentially large action space, we propose \emph{LP-Chain}, a variant that searches a cost-aware chain of feature subsets with a size that grows linearly with the number of features. While the regret upper bound is specific to the combinatorial framework, \emph{LP-Chain} empirically achieves comparable predictive performance. On synthetic data, \emph{LP-Chain} outperforms HEDGE-based BwK and deep RL-based online AFA baselines and scales favorably to more features.

0.26

Adaptive Event-Triggered Model Predictive Coverage Control with Average Update-Rate Regulation

eess.SY ·2026-10-11 ·Pengbo Zhu, Jicheng Shi, Colin N. Jones, Andreas A. Malikopoulos
benchmarktheory similarity 0.63
Abstract · Submitted to the 2027 American Control Conference (ACC)

Model predictive coverage control for multi-agent systems operates under tight communication and computation budgets, which requires it to balance the load of optimization against responsiveness to varying spatial priorities. With a fixed event threshold, the update rate varies with operating conditions, making a prescribed average update budget difficult to maintain. We propose threshold-adaptive event-triggered model predictive control (MPC), which adjusts the threshold online in proportion to the difference between the target average period and the latest inter-event interval. We prove convergence of the empirical average update period to any feasible prescribed target and establish initialization-dependent one-sided bounds on the cumulative average. A nonlinear benchmark illustrates these budget guarantees. We then apply the mechanism to coverage MPC using a task-aware statistic that captures robot motion and changes in spatial importance. In a five-agent moving-density study, the adaptive policy uses 204.7 global MPC updates on average against a target of 200, compared with 266.7 for a fixed-threshold policy using the same statistic and initial threshold. It also reduces coverage cost relative to periodic schedules with exactly matched update counts. These results demonstrate direct average-budget regulation and effective task-dependent scheduling of MPC updates.

0.36

Serve Now or Improve Later? Scheduling Self-Evolution in Online Agent Systems

cs.DC ·2026-10-05 ·Yangbo Wei, Junhong Qian, Zhen Huang, Zhenyu Su, Qifan Wang +4 more
similarity 0.63
Abstract · 21 pages, 17 figures

Online agents can improve future service by constructing reusable tools, guidance, or model states, but this work competes with current requests for the same GPUs. Exploiting idle compute for self-evolution faces a fundamental systems constraint: benefits arrive only after an artifact is published and used, while pausing evolution leaves service capacity waiting for memory release and runtime recovery. An investment worth completing may therefore be worth postponing. We present LearnSched, a state-aware scheduler that brings the reuse window of a capability and the timely return of compute into a common investment model. LearnSched incorporates candidate progress, checkpoint overhead, and the current recovery path into action values. Under the same information and capacity constraints, it uses one-step counterfactual rollout to choose progress, checkpointing, or waiting relative to a fully costed window policy. We characterize handoff costs through independent A100 component measurements and evaluate action selection in 1920 paired finite-model scenarios. When cold recovery is expensive, waiting for longer execution windows can improve net value by avoiding handoffs; in evaluated warm-recovery scenarios, the strong window policy already achieves the same value. Retaining recoverable state can shorten capacity return and reduce the need for complex evolution scheduling.

0.22