CS Paper Radar

Position: Fairness Failure in Generative Models is an Evaluation Problem

Mariia Vladimirova, Jean-Yves Franceschi, Thibaut Issenhuth

cs.LG cs.AI ·submitted 2026-08-17 14:41 UTC · v1
top venuepositioncode/data

Abstract

Despite groundbreaking advancements in generative models during the last decade, concerns about their lack of fairness, reinforcing societal inequalities and harming marginalized groups, remain under-addressed and difficult to act upon. This position paper argues that fairness failures in generative models, albeit driven by multiple factors, are ultimately stemming from an evaluation problem: fairness findings are rarely comparable across papers or actionable for deployment decisions. This paper diagnoses recurring empirical and conceptual failure modes in current practice and motivates a shift from ad-hoc bias checks to standardized, generative-specific evaluation. We propose Fairness Cards as a minimal reporting artifact that makes evaluation choices explicit (prompt families, counterfactual protocols, metrics, and refusal handling) enabling reproducibility, comparability, and accountability. We conclude with additional recommendations towards a paradigm shift in evaluation standards. Our project page can be found at https://mariiavladimirova.github.io/fairness-cards .

Comment
Accepted at ICML 2026 (Position Paper Track), cf. https://openreview.net/forum?id=5eURe8QrQo

Closest papers

Nearest neighbours by meaning across the whole index.

Fairness Theatre: Evaluating Post-Hoc Fairness Interventions in Vendor-Controlled Early Warning Systems

cs.HC cs.CYcs.LG ·2026-09-29 ·Kelly McConvey, Angelina Zhai, Rebecca Li, Shion Guha
published similarity 0.66
Abstract

Public institutions increasingly procure AI systems whose design they cannot inspect or change. In higher education, proprietary Early Warning Systems (EWS) leave colleges with few options beyond adjusting model outputs to address inequity. This raises the question of how fairness work is coordinated among vendors, institutions, advisors, and students with unequal power to change these systems? Using student records from a public college in Ontario, Canada, we evaluate six post-hoc fairness interventions on a research EWS under simulated procurement constraints. We compare fairness, accuracy, and demographic disparities, introducing error-type profiling to trace how interventions redistribute false positives and false negatives. Interventions redistributed disparities without consistently reducing them. Two implementations favored already-advantaged groups because they used group size to define disadvantage; small, marginalized groups remained poorly served. These findings show how procurement constraints and implementation choices shape the possibilities for fairness work. We call the resulting condition fairness theatre; dashboard metrics converge while groups' error burdens persist or worsen.

0.35

A Framework for Task-Agnostic Counterfactual Fairness in Transformers through Activation Steering

cs.LG cs.AIcs.CY ·2026-10-30 ·Joshua Wolff Anderson, Yanshan Wang, Shyam Visweswaran
hot direction similarity 0.65
Abstract

Bias in transformer-based models is a growing concern as these architectures become more prevalent in automated decision-making systems, such as prediction models for hiring, lending, and clinical treatment recommendation. Mechanistic interpretability (MI) is a growing body of work that measures meaningful causal effects of activations on model output by identifying human-interpretable features encoded in model activations. When such features correspond to sensitive attributes, they provide a direct way to quantify the bias those attributes introduce and to causally intervene to mitigate it. However, counterfactual fairness (CF) methods have not been systematically adapted to this setting of intervening on interpretable mechanisms in transformers. We propose a framework, Mechanistic Interpretability for Counterfactual Fairness (MICF), that utilizes MI tools to reformulate CF constraints in terms of transformer activations, unifying former approaches into a single pipeline for fairness analysis in transformers. MICF does not require a prior causal model or specific transformer architecture. We evaluate MICF on real-world tabular data and show that CF can be improved across different prediction tasks.

0.27

When Post-Processing Fairness Constraints Help and When They Harm: Evidence from Eight Cross-Domain Evaluations

cs.LG cs.CY ·2026-09-22 ·Nithin Raghava Ramachandra Narla
benchmarkcode/data similarity 0.64
Abstract · 18 pages, 6 figures, 2 tables. Code, data loaders and figures: github.com/nithinnarla/fape-fairness-ml

Fairness audits in production ML typically occur once, at deployment, on a single domain. Both fail in practice: fairness can shift after retraining or a changing user base, and interventions validated on one dataset are rarely tested across the heterogeneous domains an organization deploys. We present FAPE (Fairness Auditing for Production Environments), a four-stage framework evaluating a single post-processing intervention, Fairlearn's ThresholdOptimizer, across eight domain evaluations: criminal justice, income prediction, legal admissions, credit lending, agricultural lending, a multi-domain benchmark corpus, healthcare, and education. Each is scored on demographic parity and equalized odds difference, plus disparate impact ratio and accuracy cost where computable. Intervention effectiveness tracks baseline disparity magnitude: across model-domain pairs the constraint improved disparity in 9 of 14 high-disparity cases and worsened it in 3 of 4 near-fair ones. Each of the five high-disparity exceptions reverses under one of two measurement checks, a minimum group size or thresholds fit on held-out data. A CUSUM monitor started at deployment, tested on a simulated shift, separates constrained models that never met a 0.1 parity convention from those that met it and later regressed. A single deployment-time audit is therefore an unreliable guide, which argues for baseline-disparity screening and continuous monitoring

0.31

The Delayed Societal Cost of Unfairness

cs.LG ·2026-11-19 ·Giovani Valdrighi, Isabel Valera, Marcos Medeiros Raimundo
similarity 0.64
Abstract · Preprint, under review

Fair machine learning typically evaluates the societal costs of a decision policy -- the loss incurred by the decision-maker (DM) and the unfair distribution of costs among demographic groups of decision-subjects (DS) -- on static data, where reducing one often comes at the expense of the other. Yet decisions change the populations on which they act, and prior work has shown that fairness interventions designed on static data may fail to benefit, or even harm, the disadvantaged group over time. We investigate a complementary question: When can a policy that is Pareto-optimal on pre-deployment data be outperformed by a fairer policy once each is evaluated on the population it induces, reducing costs for one stakeholder without increasing them for the other? In a setting where qualification scores evolve in response to binary decisions and their outcomes, we show that under Rawlsian fairness, which never lowers any group's acceptance rate, placing sufficiently high weight on fairness reduces societal costs whenever accepting disadvantaged DS at the DM's break-even point leaves them in a state less costly to the DM than rejecting them. Under egalitarian fairness, which may lower the advantaged group's acceptance rate, this additionally requires the reduction in DM loss on the disadvantaged group to outweigh the increase on the advantaged group. Under both fairness notions, however, when DM loss is accuracy-based, improvements in the post-deployment population may arise from declining DS qualification, revealing that gains for the DM do not necessarily translate into better outcomes for DS. Our results show that policy-induced population dynamics can overturn apparent performance-fairness trade-offs, allowing fairer policies to reduce the delayed societal costs of decision-making.

0.19

The AR Fairness Metamodel: A Structured Framework for Fairness Measures

cs.CY cs.AIcs.LOcs.MA+1 ·2026-09-16 ·Julian Alfredo Mendez, Timotheus Kampik
code/data similarity 0.64
Abstract

This paper presents the AR fairness metamodel, a framework designed to represent, analyze, and compare different fairness scenarios. The metamodel considers key elements, such as agents, resources, and their attributes, and enables the systematic definition and comparison of various fairness measures. We provide examples involving both discrete and continuous measures, including equality, equity, group fairness, individual fairness, the Gini index, the Theil index, Jain's fairness index, and a detailed fairness measure for Australia's Child Care Subsidy. We also explore relationships among group fairness, individual fairness, and envy-freeness, supported by formal proofs. At the conceptual modeling level, our approach builds on the Tiles framework, which offers modular components that can be connected to capture diverse fairness definitions. The goal is to make AR-based fairness definitions practical and adaptable across contexts, providing a clear way to define, compare, and evaluate them. An implementation of the Tiles framework is available as an open-source tool, and can support fairness modeling and evaluation across a wide range of applications.

0.22

FairLens: Benchmarking Fairness in Vision-Language Models for High-Stakes Decision-Making

cs.CV cs.LG ·2026-09-01 ·Vahid Reza Khazaie, Ahmed Y. Radwan, Shaina Raza
benchmarkcode/data similarity 0.63
Abstract · Code and benchmark resources are available at https://vectorinstitute.github.io/fairlens

Vision-language models (VLMs) are increasingly used to make decisions from visual inputs. We introduce FAIRLENS, a benchmark and evaluation framework for measuring both the fairness and the validity of VLM responses in three high-stakes domains: hiring, legal, and healthcare. FAIRLENS pairs real face images spanning gender, race, and age groups with closed- and open-ended questions, giving more than 100K image-question pairs per model, and evaluates responses from four complementary views: demographic parity over adverse outcome rates, soundness, demographic association over unsupported roles and statuses, and bias in free-text generation. Soundness is the central validity criterion: a response is sound when it follows the evidence stated in the question and abstains when the image cannot support an answer. Evaluating eight VLMs, we find that the primary failure is unwarranted inference rather than unequal treatment. Models routinely infer qualifications, threat, illness, or professional role from a face instead of abstaining, and the weakest model does so on 99% of the questions its input cannot answer. These failures are most severe in legal and healthcare, where recognizing insufficient evidence matters most, and disparity metrics alone would miss them: parity gaps are small in absolute terms, yet when baseline adverse rates are low the same gap means one demographic group receives adverse labels several times as often as another, and a small gap can equally reflect a model that treats every group unsafely. Bias in free-text responses is only loosely coupled to multiple-choice accuracy, so correct structured answers do not imply safe generation. FAIRLENS shows that fair high-stakes VLM behavior requires similar treatment across groups and refusal to infer high-stakes attributes from appearance, and its question suite transfers to any face corpus with demographic annotations.

0.45

Fair Like Us? Auditing LLM Alignment in Resource Allocation

cs.AI cs.CYcs.GT ·2026-08-31 ·Qishen Han, Hadi Hosseini, Joshua Kavner, Samarth Khanna, Sujoy Sikdar +1 more
top venue similarity 0.63
Abstract · Accepted at EMNLP 2026 (Main Conference)

Fair allocation of scarce, indivisible resources is an important challenge in many societal problems. While there are several formal theories of fairness, no single definition can always be satisfied. As large language models (LLMs) are increasingly used to support decisions and act as agents, they raise new concerns about distributional justice: their judgments are not directly tied to any specific fairness framework and may violate key normative principles. In this work, we introduce a general method for evaluating fairness reasoning in LLMs. We study first-person fairness judgments across a broad set of models and compare them directly with human responses on matched scenarios and elicitation conditions. We find that LLMs tend to prefer stricter fairness constraints than humans, show more self-interested behavior, are sensitive to how information is framed, and are difficult to align with human judgments using fine-tuning with current datasets.

0.42

Algorithmic Fairness Over Time: Advances & Prospects

cs.LG ·2026-10-10 ·Miriam Rateike
published similarity 0.63
Abstract · Accepted at EWAF 2024, unpublished

In the presence of feedback loops, algorithmic decisions at one time step can influence observed data in subsequent steps, potentially introducing selection bias or shifts in data distribution. Research has demonstrated that ignoring these effects can lead to unfairness, which may worsen over time as the model remains deployed within society. This raises important questions of how to measure and ensure fairness of algorithmic decision-making models over time. In this short paper, we present a taxonomy that distinguishes between different types of static and temporal fairness notions in recent advancements. We then suggest prospects for future research by identifying common implicit temporal assumptions, open questions to enhance our temporal understanding of fairness, and other open challenges.

0.28