CS Paper Radar

Tilted Mondrian: Triage and Mitigation of Fine-grained Disparities in Conformal Prediction

Ziang Gao, Pengqi Liu, Archer Yi Yang, Mouloud Belbahri, Jesse C. Cresswell, Masoud Asgharian

cs.LG stat.ML ·submitted 2026-10-17 18:36 UTC · v1

Abstract

Fairness in conformal prediction is often pursued through Mondrian calibration, which provides finite-sample coverage guarantees separately across groups. These guarantees, however, are coarse: substantial disparities can remain within finer strata. This coarse-to-fine fairness issue raises two questions: how severe can such disparities be, and how can they be mitigated? Aggregate disparities can depend on how groups are distributed across strata. For severity assessment, we introduce propensity standardization to examine the effect of population composition, and show how group composition can induce disagreement between aggregate and finer-stratum comparisons. For mitigation, we develop Tilted Mondrian to improve fairness compared to group-Mondrian at a finer level without additional model training. Tilted Mondrian retains groupwise coverage guarantees and mitigates coverage and prediction set size disparities within finer strata. Finally, we derive a method-agnostic bound that quantifies how much reduction in finer-stratum coverage disparity can be achieved.

Closest papers

Nearest neighbours by meaning across the whole index.

Overcoming the Resolution Limit: Significance-Aware Regularization for Intersectional Fairness

cs.LG ·2026-11-11 ·Antonio Ferrara, André Panisson, Francesco Cozzi, Alan Perotti, Francesco Bonchi
top venuebenchmark similarity 0.66
Abstract · Accepted at NeurIPS 2026

Intersectional fairness requires evaluating models across fine-grained subgroups. However, as group definitions become more granular, the sample size per subgroup shrinks, causing fairness estimates to become increasingly unstable. Existing in-processing methods are certainty-blind: they penalize observed disparities without distinguishing statistically certified unfairness from sampling noise. This leads to fairness overfitting, where models sacrifice utility to "fix" spurious disparities in sparse regimes. To address this, we introduce the significant-excess metric family, a continuous, uncertainty-aware fairness score that measures only the portion of a subgroup's disparity exceeding what sampling variation can explain at a specific confidence level. Building on this metric, we propose SAFER (Significance-Aware Fairness Regularizer), a differentiable in-processing method that gates fairness penalties using a soft-count likelihood-ratio evidence statistic. SAFER satisfies a provable safety property: when every subgroup disparity lies well below the gate threshold, the unconstrained optimum is nearly stationary for SAFER. At that optimum, the back-propagated contribution of its significance-gated component is exponentially small, and the additional fairness contribution comes from a small ungated term with a fixed weight. Mechanism ablations across three benchmark datasets and purpose-built stress variants show that removing any component of SAFER (threshold smoothing, the significance gate, or a dense-tail term) produces the failure the theory predicts on at least one stress variant. In sparse intersectional settings, SAFER achieves a better fairness-utility Pareto frontier than the baseline methods, and it preserves utility when certified unfairness is absent.

0.24

Target-Stratified Fair Range Summaries: Improved Fair $\varepsilon$-Nets and Geometric Hitting Sets

cs.DS cs.DM ·2026-09-17 ·Mingchao Zhou, Lei Zhao, Zhipeng Cai, Zhao Zhang
theory similarity 0.62
Abstract

Compact summaries are a key tool for approximate query processing over large datasets. For range-query workloads, an $\varepsilon$-net provides a small summary that hits every sufficiently large range. However, classical $\varepsilon$-nets only guarantee range validity and do not control the group composition of the selected tuples. As a result, the summary may be range-valid but poorly representative, which can propagate imbalance to downstream query results. Motivated by recent work on fair $\varepsilon$-nets and fair geometric hitting sets \cite{dehghankar2025fair}, we study fairness-aware range summaries under prescribed target group ratios. Different from previous sample-and-repair approach, we propose a target-stratified sampling method. For demographic parity (in which the ratio of fairness is determined by group proportion), our sample size is $O(A_{\varepsilon})$, coinciding with the standard $\varepsilon$-net bound, improving previous bound of $O\!\left(A_\varepsilon\log\frac{k}{\varphi}\right)$. For custom-ratio targets (in which the ratio of fairness is determined by manually defined proportion), our sample size is $O(A_Γ)$, where $Γ$ is a parameter measuring the gap between the customized ratio and the demographic parity; we prove that this dependence on $Γ$ is unavoidable, with a worst-case lower bound of $Ω(Γ/\varepsilon)$. Using our target-stratified sampling method, we could improve the previous approximation ratio for the fair geometric hitting set problem by a logarithmic factor, and making use of this result, we could in turn improve the size of custom-ratio fair $\varepsilon$-net. Experiments on real and synthetic datasets demonstrate that our method constructs smaller fair summaries than existing approaches, scales to large datasets and fine-grained group constraints, and improves downstream range query processing.

0.19

CS-WCP: Robust Conformal Sets for LLM-Judge Traffic Shifts with Uncertain Group Proportions

cs.LG ·2026-08-23 ·Ibne Farabi Shihab, Fariya Afrin
similarity 0.59
Abstract

Prediction sets built from an LLM judge can undercover when deployment traffic changes the prevalence of task or policy groups. Weighted conformal prediction is exact under covariate shift when the density ratio is known, but group proportions must usually be estimated from finite unlabeled samples. We introduce confidence-set weighted conformal prediction (CS-WCP), which constructs simultaneous exact intervals for source and target group masses and returns the union of weighted conformal sets over every compatible ratio vector. For a fixed or independently learned finite partition, CS-WCP attains coverage at least 1-alpha-delta_w-tau_A-kappa, where tau_A measures within-cell covariate mismatch and kappa measures conditional shift. A linear endpoint rule computes the robust union in O(G|Y|) time. Across 336 constructed shared-support traffic shifts, CS-WCP reaches 0.973 mean coverage with 13 point failures, compared with 0.954 and 44 failures for source conformal prediction, at mean binary set sizes 1.74 and 1.65. On 336 natural cross-task transfers, coverage rises from 0.882 to 0.962, but mean set size reaches 1.87 and a size-matched group plug-in baseline is competitive. The method therefore supplies an auditable coverage safeguard under uncertain mixture weights; its value is conservative tail protection, not scalar probability calibration or uniformly smaller sets.

0.45

Beyond Demographic Balance: Multi-Metric and Intersectional Evaluation of Fairness in MIMIC-IV Mortality Prediction

cs.LG ·2026-10-01 ·Abdullah Al Noman, Fahmid Al Rifat, Tahrima Hashem, Syed Muhammad Ibne Zulfiker, Rishov Paul +1 more
similarity 0.59
Abstract · NEurlPS TAE workshop 2026 accepted

Fairness conclusions in clinical prediction can depend strongly on both the metrics reported and the demographic resolution at which performance is evaluated. We revisit these evaluation choices for ICU mortality prediction on MIMIC-IV, comparing predictive-utility and subgroup-error metrics across several fairness interventions. As a complementary case study, we introduce a lightweight adaptation strategy that jointly balances ethnicity--gender--insurance representation without conditioning on mortality outcomes, allowing demographic representation balancing to be examined separately from outcome-conditioned or direct error-rate interventions. We evaluate its behavior at both marginal and corresponding three-way intersectional subgroup levels, while accounting for the statistical support of finer-grained estimates. The results show that interventions can receive substantially different assessments across accuracy/AUROC, sensitivity, and false-positive rate, and that marginal demographic summaries can conceal heterogeneous error profiles within their constituent intersections, including among larger subgroups. These findings highlight the importance of evaluating fairness interventions at both complementary metric and subgroup resolutions, while accounting for the intervention target and the reliability of subgroup estimates.

0.15

Surrogate Losses for Fairness-Aware Classification: Fisher Consistency and Calibration

cs.LG stat.ML ·2026-11-01 ·Changcheng Song, Yi Yang, Xiangyu Chang, Xiaoling Lu, Yufeng Liu
hot direction similarity 0.58
Abstract

Fairness-aware classification seeks to balance predictive accuracy and fairness across demographic groups. Although surrogate losses facilitate the optimization of fairness-aware objectives, it remains unclear when minimizing these losses yields optimal decisions under the original fairness-aware risk. This paper investigates the statistical validity of surrogate-based fairness-aware classification under demographic parity and equal opportunity, considering both attribute-blind and attribute-aware settings. We introduce fairness-aware Fisher consistency and characterize it through conditions on the derivatives of the surrogate losses. The characterization shows that consistency depends not only on the individual loss functions but also on their interaction within the fairness-regularized objective. We further establish a calibration inequality that relates the excess fairness-aware risk to the excess surrogate risk, providing a quantitative connection between surrogate optimization and the original classification objective. Numerical experiments illustrate how deviations from the consistency conditions affect fairness-aware excess risk and how the regularization parameter influences the well-posedness of surrogate optimization. An application to consumer credit default prediction demonstrates that appropriately calibrated fairness regularization can substantially reduce demographic disparities while maintaining predictive performance.

0.25

Stochastic Grouping Conformal Prediction for Effective Subgroup Reliability

cs.LG cs.AI ·2026-10-08 ·Meihui Zhong, Wenxin Tai, Ting Zhong, Fan Zhou
theory similarity 0.57
Abstract · 9 pages

Conformal prediction offers a distribution-free coverage guarantee, making it especially attractive for clinical applications. Standard conformal prediction, however, provides such guarantees only at the population level, and its prediction sets can exhibit coverage disparities across clinically important subgroups. A natural remedy is to calibrate within predefined groups. However, this can require access to sensitive subgroup attributes and is prone to a worst-group bottleneck: protecting the most difficult subgroup can inflate prediction sets for all, increasing cognitive burden on decision makers. To this end, we propose Stochastic Grouping Conformal Prediction (SGCP), a conformal framework for subgroup-reliable uncertainty quantification. It learns a stochastic grouping map that allows each sample to draw calibration information from others with similar calibration behavior, yielding a local score law that boosts reliability across subpopulations. We prove that SGCP retains the standard coverage guarantee. Experiments on synthetic and real-world benchmarks show that it consistently reduces subgroup coverage gaps while achieving smaller or comparable prediction set sizes relative to existing baselines.

0.18

Towards stratified sampling for redistricting plans

cs.CY physics.soc-phmath.PR ·2026-09-16 ·Zijian Wang, Gregory J. Herschlag, Joon-Hyeok Yim, Jonathan C. Mattingly, Anna C. Gilbert
similarity 0.57
Abstract

Rapid algorithmic developments have accelerated the sampling of redistricting ensembles (balanced graph partitions), yet evaluating rare events and sampling complex target measures remains a core challenge due to the high-dimensional and combinatorial nature of the phase space. We address a prerequisite for stratified sampling on this space: constructing and diagnosing candidate strata with suitable coverage and overlap. We build a grammar on observed plans by clustering districts into representative ``letters'' and using them to form plan-level ``words.'' A partition of unity over these words gives a soft assignment of plans to strata and allows us to estimate stratum masses and an overlap-induced flux matrix. We demonstrate this computational pipeline using real-world congressional redistricting data from Connecticut and examine how strata learned from one target distribution behave under related distributions. The resulting construction provides a foundation for future stratified sampling on spaces of redistricting plans or balanced graph partitions. We do not implement a complete stratified sampler here; evaluating whether the proposed strata improve sampling efficiency or reduce estimator variance is left for future work.

0.27

A Comprehensive View of Fairness through Distributional Stability

cs.LG cs.AI ·2026-09-29 ·Gayane Taturyan, Charlotte Laclau, Stephan Clémencon
similarity 0.57
Abstract

We view fairness as a property of distributional stability. Rather than assessing a predictor under a fixed data distribution, we study how its predictions change under perturbations that modify the composition of protected groups. A predictor is fair if it remains stable under such shifts. Under this perspective, several classical notions of fairness arise as stability with respect to specific perturbations, with the associated unfairness gap given by a Lipschitz constant of a prediction-rate functional. This formulation also yields guarantees that hold uniformly over a range of demographic compositions at test time, without requiring knowledge of the deployment distribution. It leads to a learning procedure based on convex combinations of reweighted predictors, formulated as a second-order cone program, for which we establish generalization bounds. Experiments on standard benchmarks illustrate the approach.

0.41