CS Paper Radar

Conformal Robustness in Prediction-Driven Decision-Making

Lingjie Zhao, Hansheng Jiang, Wei Qi

cs.LG stat.MLmath.OC ·submitted 2026-09-19 18:43 UTC · v1

Abstract

Modern prediction-driven decision systems often rely on black-box predictors, but a point forecast alone does not provide the uncertainty scale required for robust downstream decision-making. We build a score-calibrated robustness framework that converts any fixed point predictor into a decision-relevant uncertainty representation through distribution-free conformal calibration. We use the conformal score, rather than a particular uncertainty set, as the primitive unit of robustness. The same score determines coverage-calibrated uncertainty sets for reliability-based robust optimization and normalizes target violations in a target-oriented formulation, Conformal Robust Satisficing. This formulation induces a conformal fragility measure that quantifies how rapidly performance deteriorates as the realized parameter departs from the forecast on the conformal score scale. For objective-uncertainty problems under standard convexity and duality conditions, we show that the reliability-based and target-oriented formulations parameterize the same score-calibrated robust decision frontier. This equivalence yields a data-driven mapping between reliability levels and acceptable targets and characterizes the marginal cost of robustness. Synthetic experiments validate the theoretical guarantees and illustrate the reliability-target correspondence. A real-data online-grocery case study demonstrates how the interface combines deep-learning demand forecasts with tractable inventory optimization, thereby improving reliability and reducing operational costs. Overall, our work shows that conformal scores endow fixed black-box predictors with an interpretable uncertainty scale for downstream decision-making while enabling reliability guarantees, acceptable-target selection, and fragility analysis within a unified framework.

Closest papers

Nearest neighbours by meaning across the whole index.

A Unified Perspective on Conformal Prediction and Wasserstein Distributionally Robust Optimization for Uncertainty Quantification

cs.LG math.OCeess.SY ·2026-08-30 ·Kehan Long, Yiqi Zhao, Pol Mestres, Lars Lindemann, Nikolay Atanasov +1 more
similarity 0.72
Abstract

Uncertainty quantification from finite data is central to machine learning, optimization, and automation systems, where decisions must remain reliable under limited samples and test-time distribution shift. Conformal prediction (CP) and distributionally robust optimization (DRO) offer two complementary approaches: CP constructs data-dependent prediction sets with distribution-free finite-sample validity under exchangeability, while DRO optimizes worst-case performance over an ambiguity set around an empirical distribution. We develop a unified probabilistic perspective on CP and DRO by viewing both as ways to turn finite calibration data into a data-dependent quantile estimator that a test score falls below with high probability. From this perspective, CP and DRO correct the empirical quantile along two coordinates of the same family of estimators: CP inflates the quantile level, whereas DRO shifts the quantile value through an ambiguity radius. Both methods provide the same calibration-conditional guarantee for the true distribution, requiring the target coverage to hold with high probability over the calibration sample. Their constructions differ, however: CP uses a closed-form, distribution-free level correction, while DRO uses a value-space correction whose certified radius depends on properties of the unknown distribution and additionally guarantees coverage uniformly over the ambiguity set. This distinction emerges in the tails of the score distribution. Because CP relies on sparse upper-tail order statistics of the calibration samples, its level inflation barely moves the estimator when those samples are dense near the target quantile but overshoots when they are sparse, whereas a well-chosen DRO radius corrects in value space and may avoid this overshoot.

0.30

Safety-Critical Control under Uncertainty via Adaptive Conformal Quantile Prediction Intervals

cs.RO ·2026-09-20 ·Hao Zhou, Yanze Zhang, Yiwei Lyu, Wenhao Luo
published similarity 0.71
Abstract · 8 pages, accepted to CDC 2026

Safety-critical control under uncertainty requires uncertainty representations that are both statistically valid (for certifiable performance) and compatible with enforceable safety constraints. However, existing methods often assume particular distributions of uncertainty for provable safety guarantees or establish symmetric and input-agnostic prediction intervals for robust safety, which can lead to misaligned or overly conservative safety constraints in control synthesis. In this paper, we introduce a novel safe control framework with adaptive uncertainty quantification that constructs calibrated and state-dependent prediction intervals to enable high-probability safety guarantees, while improving constrained control performance. The framework leverages adaptive conformal prediction (ACP) and extends it with conformal quantile regression (CQR) to capture distribution-free, asymmetric uncertainty intervals with certifiable probabilistic coverage, and integrates the resulting uncertainty sets into a probabilistic control barrier function formulation to enforce robust safety with reduced conservativeness. This yields uncertainty-aware safe control constraints that can be incorporated within a model predictive control(MPC) framework to provide provably safe behaviors with high probability. Simulation and theoretical results are provided to demonstrate the effectiveness of our approach.

0.28

Beyond Prediction Residuals: Conformal Regret Calibration for Robust Contextual Linear Optimization

cs.LG stat.ML ·2026-11-09 ·Lingjie Zhao, Hansheng Jiang, Wei Qi
theory similarity 0.70
Abstract

Predict-then-optimize systems convert contextual forecasts into decisions, yet conformal robust optimization typically calibrates prediction residuals rather than downstream loss. For linear optimization, this parameter-space geometry can be badly misaligned with decision risk: large objective perturbations may preserve the optimizer, while small boundary-crossing perturbations may cause substantial regret. We introduce Conformal Regret Sets, a post-hoc, model-agnostic framework that conformalizes an optimization-derived regret surrogate and uses its calibrated level set for minimax-regret refinement. We first show that directly conformalizing exact regret, although sufficient to certify the nominal predict-then-optimize decision, yields a degenerate robust problem: absent an exogenous bounded support, every alternative decision has unbounded worst-case regret. We therefore use SPO+, a pointwise upper bound on exact regret, as the conformal score, obtaining distribution-free finite-sample marginal regret guarantees under exchangeability. For contextual linear programs, the resulting set decomposes into polyhedral pieces aligned with the normal fan of the feasible polytope. A vertex is active if and only if its predicted suboptimality gap is at most half the calibrated threshold $η$, enabling local, output-sensitive generation of all relevant pieces. Moreover, if $Γ$ is the predicted gap to the closest competing vertex, the nominal and refined decisions admit the sharper worst-case-regret certificate $(η-2Γ)_+$, with an exact early exit when $η\leq 2Γ$. Experiments on contextual fractional knapsack validate the geometry and certificates: continuous refinement reduces the $0.95$-quantile and $\operatorname{CVaR}_{0.95}$ of regret by $9.1\%$ and $13.1\%$ relative to nominal PTO, while vertex-restricted refinement changes only $10.0\%$ of nominal decisions.

0.08

Selected-Risk Calibration for Feasibility at Returned Decisions

cs.LG stat.ML ·2026-11-05 ·Raghav Thind, Youran Sun, Ling Liang, Haizhao Yang
similarity 0.67
Abstract · 20 pages, 3 figures

When an optimization problem uses a surrogate constraint, a decision that is feasible for the surrogate need not be feasible for the true constraint. We propose Selected-Risk Calibration (SRC), a framework that calibrates a learned constraint against the optimizer's returned decisions for objectives drawn from a given distribution. SRC guarantees that, with probability at least $1-δ$, every margin it certifies on held-out objectives has selected risk, the probability that the returned decision is infeasible, of at most $γ$. We also show that the random-input coverage guarantee of split conformal prediction does not bound the selected risk, and that likelihood-ratio reweighting for feedback covariate shift does not apply when the learned constraint is active at the returned decision. Our experiments show that a conformal margin covering about $90\%$ of random inputs covered as few as $17\%$ of the returned decisions, and every certified margin had a violation rate below its target on unseen objectives. Moreover, at equal cost, SRC's correction had fewer or equal infeasible solutions to the baselines in 57 of 62 comparisons. The SRC framework applies to any setting in which a surrogate model chooses a decision whose feasibility can be verified for a sample of instances.

0.23

CoCoNet: Convex Conformal Network for Multi-Dimensional Uncertainty Regions

cs.LG stat.ML ·2026-10-16 ·Alireza Moradi, Thomas Bruys, Pascal Van Hentenryck, Reza Zandehshahvar
similarity 0.65
Abstract

Conformal prediction (CP) provides distribution-free coverage guarantees under exchangeability, but the efficiency of its prediction regions depends critically on the choice of the conformity score. This is particularly important in multivariate regression, where dependencies across output dimensions induce complex joint uncertainty structures. This paper introduces CoCoNet (Convex Conformal Network) for learning convex conformity scores that induce flexible, data-adaptive prediction regions. CoCoNet learns the score from held-out data by optimizing prediction-region efficiency while enforcing convexity with respect to the residual. CoCoNet then applies split-conformal calibration to retain finite-sample coverage guarantees. The paper presents two variants of the method: CoCoNet$_{\mathrm{Spline}}$, based on piecewise-linear scores, and CoCoNet$_{\mathrm{ICNN}}$, based on Input Convex Neural Networks. Experiments compare CoCoNet with other CP baselines over synthetic and real-world datasets with output dimensions ranging from 2 to 16. The results demonstrate the flexibility of CoCoNet in learning different response types. Across 8 models and 30 multi-dimensional regression tasks, CoCoNet$_\text{ICNN}$ and CoCoNet$_{\mathrm{Spline}\text{-}\ell_1}$ achieve the best efficiencies with average ranks of 1.45 and 2.75, respectively. In particular, CoCoNet$_\text{ICNN}$ ranks first on 22 tasks out of 30 in terms of efficiency.

0.11

What "Form'' of Uncertainty Does Conformal Prediction Quantify?

cs.LG ·2026-10-17 ·Soroush H. Zargarbashi, Mohammad Sadegh Akhondzadeh, Rebekka Burkholz, Aleksandar Bojchevski
similarity 0.65
Abstract

Conformal prediction (CP) is a post-hoc calibration method that returns prediction sets with distribution-free coverage guarantees, and the size of the output set is often interpreted as a notion of uncertainty. We show that the difference in set size between full and split CP is directly connected to Bayesian posterior variance, which is the epistemic uncertainty of the model. Essentially, this difference in set size is an aggregation of two opposite effects: (1) better model accuracy and therefore smaller set size achieved by training over extra calibration points, and (2) the "formability" of the model at test point towards every possible label. The latter effect vanishes with stronger models or more calibration data (as epistemic uncertainty is known for that behavior). It turns out that the model's formability itself (without the quantization effect of prediction sets, and through estimating the posterior variance) is a useful tool to quantify the epistemic uncertainty. In our empirical evaluation it performs on par with the state of the art in many setups.

0.11

Split Conformal Prediction with Label-Shift-Adjusted Bayesian Scores

cs.LG ·2026-09-11 ·Hyeonsu Lee, Juyeon Kim, Erkhembayar Jadamba, Seungjin Choi, Hyunjin Shin
publishedhighly cited similarity 0.62
Abstract · 2nd Workshop on Epistemic Intelligence in Machine Learning (EIML@ICML 2026),

Conformal prediction provides distribution-free uncertainty quantification under exchangeability. However, this assumption is violated by label shift, where the marginal distribution of labels changes while the conditional distribution of inputs given labels remains stable. Under such shifts, standard conformal procedures no longer maintain their intended coverage behavior. Existing approaches address this via importance weighting. They pair the reweighting with residual-based nonconformity scores that ignore predictive uncertainty. The resulting intervals have uniform width. Bayesian conformal methods produce adaptive intervals by leveraging predictive distributions. They evaluate conformity under the source predictive, which is misaligned with the target domain under label shift. We propose the \emph{Label-Shift-Adjusted Bayesian Score} (LSA score), a nonconformity score derived from a posterior predictive tilting identity. This identity shows that the target predictive is an importance-weighted transformation of the source predictive. We use it to derive a direct correction to the Bayesian score. We evaluate the method on molecular property prediction under controlled label shift. The LSA score consistently yields shorter intervals than residual-based and source-based Bayesian scores. Coverage in the target domain remains comparable. Under stronger shift, all methods incur some coverage loss due to pseudo-label-based density-ratio estimation. The LSA score is defined for any source predictive with a tractable log-density. We instantiate it with Bayesian Ridge Regression, where the correction admits a closed form.

0.54

Sequential Confidence Sets for Coverage-Constrained Conformal Model Selection

cs.LG stat.ML ·2026-09-22 ·Jing Li, Haibin Zhu
similarity 0.62
Abstract

Modern conformal forecasting systems often maintain several adaptive pipelines that differ in base forecasters, conformity scores, calibration windows, and update rules. Comparing them is difficult because coverage is a hard constraint, whereas efficiency should be optimized only among feasible pipelines. We formulate this problem as sequential inference for a stochastic constrained argmin. At each time, the target is the set of minimum-cost pipelines satisfying multiple prefix-average conditional miscoverage constraints. We introduce Coverage-Constrained Sequential Model Confidence Sets (CC-SMCS), which separate certifiably feasible, possibly feasible, and possibly constrained-optimal pipelines. Using simultaneous martingale confidence sequences, CC-SMCS projects a rectangular confidence region onto the constrained argmin and admits an exact closed-form rule. With probability at least $1-δ$, it contains every constrained-optimal pipeline simultaneously over all times. This finite-sample guarantee requires no stationarity or mixing assumptions and remains valid under data-dependent stopping. We also establish an impossibility result for safe certification at the coverage boundary and extend the construction to delayed multi-horizon feedback and outcome-dependent efficiency objectives.

0.32