CS Paper Radar

The Delayed Societal Cost of Unfairness

Giovani Valdrighi, Isabel Valera, Marcos Medeiros Raimundo

cs.LG ·submitted 2026-11-19 12:39 UTC · v1

Abstract

Fair machine learning typically evaluates the societal costs of a decision policy -- the loss incurred by the decision-maker (DM) and the unfair distribution of costs among demographic groups of decision-subjects (DS) -- on static data, where reducing one often comes at the expense of the other. Yet decisions change the populations on which they act, and prior work has shown that fairness interventions designed on static data may fail to benefit, or even harm, the disadvantaged group over time. We investigate a complementary question: When can a policy that is Pareto-optimal on pre-deployment data be outperformed by a fairer policy once each is evaluated on the population it induces, reducing costs for one stakeholder without increasing them for the other? In a setting where qualification scores evolve in response to binary decisions and their outcomes, we show that under Rawlsian fairness, which never lowers any group's acceptance rate, placing sufficiently high weight on fairness reduces societal costs whenever accepting disadvantaged DS at the DM's break-even point leaves them in a state less costly to the DM than rejecting them. Under egalitarian fairness, which may lower the advantaged group's acceptance rate, this additionally requires the reduction in DM loss on the disadvantaged group to outweigh the increase on the advantaged group. Under both fairness notions, however, when DM loss is accuracy-based, improvements in the post-deployment population may arise from declining DS qualification, revealing that gains for the DM do not necessarily translate into better outcomes for DS. Our results show that policy-induced population dynamics can overturn apparent performance-fairness trade-offs, allowing fairer policies to reduce the delayed societal costs of decision-making.

Comment
Preprint, under review

Closest papers

Nearest neighbours by meaning across the whole index.

A Comprehensive View of Fairness through Distributional Stability

cs.LG cs.AI ·2026-09-29 ·Gayane Taturyan, Charlotte Laclau, Stephan Clémencon
similarity 0.66
Abstract

We view fairness as a property of distributional stability. Rather than assessing a predictor under a fixed data distribution, we study how its predictions change under perturbations that modify the composition of protected groups. A predictor is fair if it remains stable under such shifts. Under this perspective, several classical notions of fairness arise as stability with respect to specific perturbations, with the associated unfairness gap given by a Lipschitz constant of a prediction-rate functional. This formulation also yields guarantees that hold uniformly over a range of demographic compositions at test time, without requiring knowledge of the deployment distribution. It leads to a learning procedure based on convex combinations of reweighted predictors, formulated as a second-order cone program, for which we establish generalization bounds. Experiments on standard benchmarks illustrate the approach.

0.41

Surrogate Losses for Fairness-Aware Classification: Fisher Consistency and Calibration

cs.LG stat.ML ·2026-11-01 ·Changcheng Song, Yi Yang, Xiangyu Chang, Xiaoling Lu, Yufeng Liu
hot direction similarity 0.65
Abstract

Fairness-aware classification seeks to balance predictive accuracy and fairness across demographic groups. Although surrogate losses facilitate the optimization of fairness-aware objectives, it remains unclear when minimizing these losses yields optimal decisions under the original fairness-aware risk. This paper investigates the statistical validity of surrogate-based fairness-aware classification under demographic parity and equal opportunity, considering both attribute-blind and attribute-aware settings. We introduce fairness-aware Fisher consistency and characterize it through conditions on the derivatives of the surrogate losses. The characterization shows that consistency depends not only on the individual loss functions but also on their interaction within the fairness-regularized objective. We further establish a calibration inequality that relates the excess fairness-aware risk to the excess surrogate risk, providing a quantitative connection between surrogate optimization and the original classification objective. Numerical experiments illustrate how deviations from the consistency conditions affect fairness-aware excess risk and how the regularization parameter influences the well-posedness of surrogate optimization. An application to consumer credit default prediction demonstrates that appropriately calibrated fairness regularization can substantially reduce demographic disparities while maintaining predictive performance.

0.25

Algorithmic Fairness Over Time: Advances & Prospects

cs.LG ·2026-10-10 ·Miriam Rateike
published similarity 0.65
Abstract · Accepted at EWAF 2024, unpublished

In the presence of feedback loops, algorithmic decisions at one time step can influence observed data in subsequent steps, potentially introducing selection bias or shifts in data distribution. Research has demonstrated that ignoring these effects can lead to unfairness, which may worsen over time as the model remains deployed within society. This raises important questions of how to measure and ensure fairness of algorithmic decision-making models over time. In this short paper, we present a taxonomy that distinguishes between different types of static and temporal fairness notions in recent advancements. We then suggest prospects for future research by identifying common implicit temporal assumptions, open questions to enhance our temporal understanding of fairness, and other open challenges.

0.28

Position: Fairness Failure in Generative Models is an Evaluation Problem

cs.LG cs.AI ·2026-08-17 ·Mariia Vladimirova, Jean-Yves Franceschi, Thibaut Issenhuth
top venuepositioncode/data similarity 0.64
Abstract · Accepted at ICML 2026 (Position Paper Track), cf. https://openreview.net/forum?id=5eURe8QrQo

Despite groundbreaking advancements in generative models during the last decade, concerns about their lack of fairness, reinforcing societal inequalities and harming marginalized groups, remain under-addressed and difficult to act upon. This position paper argues that fairness failures in generative models, albeit driven by multiple factors, are ultimately stemming from an evaluation problem: fairness findings are rarely comparable across papers or actionable for deployment decisions. This paper diagnoses recurring empirical and conceptual failure modes in current practice and motivates a shift from ad-hoc bias checks to standardized, generative-specific evaluation. We propose Fairness Cards as a minimal reporting artifact that makes evaluation choices explicit (prompt families, counterfactual protocols, metrics, and refusal handling) enabling reproducibility, comparability, and accountability. We conclude with additional recommendations towards a paradigm shift in evaluation standards. Our project page can be found at https://mariiavladimirova.github.io/fairness-cards .

0.48

Is invariance all you need for algorithmic fairness? Removing demographic information can create new bias

cs.LG cs.AI ·2026-09-25 ·Aditya Parikh, Eike Petersen, Stella Frank, Enzo Ferrante, Melanie Ganz +1 more
similarity 0.64
Abstract · An earlier version is available as a non-peer reviewed preprint, cited in the paper. This submission supersedes it with consent of all autho…

Encoded demographic information in internal model representations is a commonly assumed risk factor for algorithmic bias, with demographic representation invariance often being touted as the ideal state. However, while demographic shortcut learning is a genuine threat, some degree of encoding is necessary when demographics correlate with target labels. Here, we show, mathematically and empirically, that enforcing demographic invariance can actually hamper bias mitigation and even create new biases. We distinguish marginal from class-conditional representation invariance, and show that they imply the standard group fairness notions of demographic parity and equalized odds, respectively. We evaluate the effects on predictive performance and fairness of enforcing both invariance types, both theoretically and empirically across five tabular and two chest X-ray imaging datasets. Our findings support our mathematical argument that demographic representation invariance is neither desirable nor sufficient for fairness.

0.07

Robbing Peter to Pay Paul: Cross-Attribute Externalities of Fairness Interventions

cs.LG stat.ML ·2026-11-10 ·Fernandes Machado Agathe, Jamotton Charlotte, Charpentier Arthur
hot direction similarity 0.63
Abstract

Fairness interventions often targets disparities associated with a single sensitive attribute in binary classification tasks. However, when multiple sensitive attributes are associated with a data subject, improving fairness with respect to one attribute does not necessarily improve it across all attributes: it may exacerbate disparities associated with other sensitive attributes, including those that are unavailable to the practitioner. We refer to these unintended effects as cross-attribute externalities of fairness. We study how fairness with respect to one sensitive attribute varies when an initially unfair classification or regression model is corrected with respect to another attribute. We characterize these externalities using Wasserstein barycentric disparities and develop a distributional framework that captures changes in both marginal and joint fairness measures. This highlights a limitation of conventional fairness evaluation, in which the success of an intervention is assessed mainly for the attribute it targets. In a jointly Gaussian model, we derive explicit trajectories for marginal and joint disparities and identify conditions under which an intervention eliminates disparities with respect to its target attribute while increasing disparities associated with another attribute. We also derive sharp partial-identification bounds on the resulting externalities when a sensitive attribute is unavailable and extend our results to categorical sensitive attributes. We illustrate these externalities using classification and regression real-data: in the classification experiments, interventions increase unfairness with respect to the hidden attribute in the majority of scenarios across more than half of the datasets considered; in the regression experiments on the ACSIncome dataset, correcting for race reduces sex disparities, whereas subsequently correcting for age increases them.

0.23

Fairness Theatre: Evaluating Post-Hoc Fairness Interventions in Vendor-Controlled Early Warning Systems

cs.HC cs.CYcs.LG ·2026-09-29 ·Kelly McConvey, Angelina Zhai, Rebecca Li, Shion Guha
published similarity 0.63
Abstract

Public institutions increasingly procure AI systems whose design they cannot inspect or change. In higher education, proprietary Early Warning Systems (EWS) leave colleges with few options beyond adjusting model outputs to address inequity. This raises the question of how fairness work is coordinated among vendors, institutions, advisors, and students with unequal power to change these systems? Using student records from a public college in Ontario, Canada, we evaluate six post-hoc fairness interventions on a research EWS under simulated procurement constraints. We compare fairness, accuracy, and demographic disparities, introducing error-type profiling to trace how interventions redistribute false positives and false negatives. Interventions redistributed disparities without consistently reducing them. Two implementations favored already-advantaged groups because they used group size to define disadvantage; small, marginalized groups remained poorly served. These findings show how procurement constraints and implementation choices shape the possibilities for fairness work. We call the resulting condition fairness theatre; dashboard metrics converge while groups' error burdens persist or worsen.

0.35

Fairness under Prediction Error in Predict-then-Optimize Pipelines

cs.LG cs.CY ·2026-10-13 ·Hilda Adwubi Osei
similarity 0.62
Abstract · 11 pages, 1 figure

Fairness constraints in predict-then-optimize pipelines are necessarily enforced on predicted outcomes $\hat{y}$, since true outcomes $y$ are unobserved at decision time. We study whether a demographic parity constraint enforced this way continues to hold against $y$, and how the resulting violation scales with prediction error. Using a budget-constrained allocation problem solved as a linear program, we show empirically that the true-outcome fairness gap grows with prediction error $σ$ and saturates, well fit by $c(1 - e^{-σ/τ})$ with $c = 0.1545 \pm 0.0103$ and $τ= 3.9045 \pm 0.3745$; its mean is invariant to sample size while its variance decays at the standard $O(1/\sqrt{n})$ rate. We derive a two-part theoretical bound: a closed-form local sensitivity result showing the gap's first-order response to prediction error depends on only two individuals per group, validated against the solver to within three significant figures; and a universal, error-independent ceiling following from the boundedness of the decision variable, confirmed consistent with but looser than the empirical saturation level. On US Census income data, enforcing demographic parity on a trained predictor's output reduces an unconstrained disparity in true income to roughly one-third of its original size, leaving 34.4% $\pm$ 3.3% (California) and 35.7% $\pm$ 1.8% (Texas) of the disparity intact and undetectable from the predictions alone -- consistent not only in mechanism but in approximate magnitude across two states with different demographics and income distributions, and stable across ten independent train/test splits in each. A fairness constraint that appears fully satisfied at decision time can mask a substantial, systematic violation of the outcome it was meant to protect.

0.20