CS Paper Radar

Queering the Audits: Community-Based Auditing of AI Harms to Queer Communities

Organizers of QueerInAI, :, A Pranav, Alissa A. Valentine, Alex Markham, Beckett LeClair, Tereza Blazkova, Ekaterina Kornilitsina, Sofie H. Bruun, Gerasimos Spanakis, Anne Lauscher

cs.CL cs.CY ·submitted 2026-10-30 20:26 UTC · v1
published

Abstract

AI systems embed majority-group defaults into training data, evaluation metrics, and category definitions, producing documented harms for queer communities including erasure, misclassification, and discrimination. Standard technical audits often rely on aggregate measures and cannot detect harms that be come visible only through the lived experience of affected communities. We conducted a participatory auditing workshop at EurIPS 2025 where 16 queer community members audited four case studies using the 4Cs harm taxonomy (Content, Conduct, Contact, Contract) applied across the AI lifecycle. Participants used structured worksheets and plenary synthesis to classify harms and trace them to their origins in the development pipeline. Across all four cases, participants traced harms to problem definition and data collection, and they identified contractual structures that extract value from vulnerable populations while providing minimal recourse. These findings illustrate that community-informed auditing surfaces concrete, identity-specific harms that aggregate evaluation methods risk overlooking.

Comment
Published in Identity Aware AI workshop at LREC 2026
Journal ref
In Proceedings of the Second Workshop of Identity Aware AI at 2026, pages 66-76, Palma de Mallorca, Spain. European Language Resources Association
DOI
10.63317/5acrwqmesyw4

Closest papers

Nearest neighbours by meaning across the whole index.

Towards participatory speech dataset curation: A queer case study and conceptual framework

cs.AI cs.CY ·2026-09-21 ·Brooklyn Sheppard, Anaelia Ovalle, Adina Williams, Levent Sagun
top venue similarity 0.66
Abstract · Accepted at Interspeech 2026

In this paper, we motivate the need for a participatory speech dataset creation framework through a case study of the LGBTQIA+, or queer, community - a community with documented concerns about AI and reported harms, including attempts to develop 'gaydar' technologies that purportedly identify individuals as queer. We review common speech data collection practices, why these methods may be unsuitable for engaging with queer speakers, and discuss previous efforts in participatory AI with queer community engagement, as well as participatory endeavours specific to speech data collection for other marginalized communities. From this review, we develop a conceptual framework for participatory speech data curation by, for, and with marginalized communities drawing on insights from co-design and knowledge sharing. We propose a framework comprising overlapping and two-way processes of defining a community, project formulation, modes of participation, and personal autonomy.

0.43

Queer inclusion in speech datasets: An audit and taxonomy of practical tensions

cs.AI ·2026-09-21 ·Brooklyn Sheppard, Anaelia Ovalle, Adina Williams, Levent Sagun
top venue similarity 0.63
Abstract · Accepted at Interspeech 2026

In this paper, we examine speech datasets for their inclusion of LGBTQIA+, or queer, voices and provide a taxonomy of tensions to better understand why there is a lack of such voices in current speech technology datasets. Through an audit of six diverse speech datasets, we find that measurable queer representation is low (0-1.4% of speakers) - insufficient for robust disparity measurement. We take this community as a case study to consider what challenges and tensions are associated with collecting speech data from marginalized communities. For comparison, we audit an additional two datasets from the speech sciences that were created by, for, and with the queer community. We note that many customs in speech dataset collection efforts in AI and speech technology research may conflict with values emphasized in participatory approaches with marginalized communities, and provide a taxonomy describing these tensions.

0.32

Terms of Care: Designing for Participation in Data Governance by Disability Communities

cs.HC cs.AIcs.DL ·2026-11-11 ·Madiha Zahrah Choksi, Reeda Shimaz Huda, Rita Faia Marques, Anja Thieme, Cecily Morrison
similarity 0.61
Abstract · 26 pages, 3 figures, 3 tables, under review

Improving disability representation in AI-generated media requires community-specific datasets, yet openly sharing such data can expose communities to misuse, decontextualization, and exploitation. We examine how disability communities can participate in dataset governance through context-sensitive licensing. Working with three disability advocacy organizations, we conducted semi-structured interviews, an educational webinar and modular license activity to promote community participation in data stewardship. Our findings show that governance in these high-context communities relies on organizational memory (especially situated harms); both formal authority and informal relational practices; communication; and resulting trust. The modular license activity made organizational data choices more accessible and values-led, allowing communities to adapt Creative Commons clauses to express permitted uses, sector-specific restrictions, safeguarding requirements, and commitments to respectful representation. Yet, concerns remain about monitoring and enforcing appropriate data use. We contribute (1) an empirical account of governance tensions, (2) a practical modular approach, and (3) design implications for AI data platforms that support relational, context-sensitive governance beyond standardized, one-size-fits-all frameworks.

0.41

Move Fast and Mend Things: Keeping Up with Evolving AI Harms Using Social Media Commentary

cs.HC cs.CY ·2026-10-06 ·Jacqueline Rowe, Animesh Srivastava, Sai Teja Peddinti, Seliem El-Sayed, Nina Taft
similarity 0.60
Abstract

The rapid deployment of AI systems has created socio-technical, psychological, and operational harms that can elude ex-ante threat modelling and ex-post incident tracking. We introduce an LLM-assisted thematic analysis pipeline to dynamically detect, categorise, and track emerging AI harms from large-scale social media data. Applying it to 5.7 million Reddit post summaries over 18 months (01/2025 to 06/2026), we curate and release a dataset of 575,000 AI harm-related posts and a bottom-up AI harm taxonomy of 12 categories and 47 subnodes. The taxonomy reliably covers established expert-defined risks while surfacing granular harms that top-down frameworks overlook, such as distinct forms of AI privacy violations. Temporal analysis surfaces evolving user-centric harms, such as agentic privacy and security breaches, premature AI adoption in the workplace, and grief from AI companion discontinuation. Our pipeline shortens harm-detection timelines and hereby complements efforts towards more participatory and responsive AI governance.

0.07

Too Big to Inspect? Ethically Auditing Large Datasets Through Small Samples

cs.AI cs.DB ·2026-11-07 ·Giovana Marchetti, Jhessica Silva, Tiago Torrent, Carlos Caetano, Sandra Avila
similarity 0.60
Abstract · 12 pages, 6 figures

Big techs and researchers increasingly claim that their training datasets are too large to inspect for bias and ethical problems. However, in the race to build ever-large models, they have relied on automated web scraping and no human curation. Datasets built this way absorb the stereotypes, exclusionary norms, and historical prejudices of their sources, and the models trained on them can reproduce and amplify those patterns. Because manually auditing billions of samples is humanly impossible, we need practical, effective methods to identify biases in these datasets. This paper studies the feasibility of ethically auditing large datasets through small samples and proposes a practical method for identifying structural biases by applying AI Ethics Tools (AIETs) to random samples. We applied Datasheets for Datasets, Data Statements, Data Nutrition Labels, and Data Cards to a 1% sample of Framed Multi30K, and Datasheets for Datasets to two 0.00006% samples of relaion2B-en-research-safe. Our results show that evaluating small samples with AIETs can reveal ethical problems, structural biases, and technical inconsistencies. Although industry practice assumes that identifying problems requires analyzing the entire dataset, we show that a random subset supports a viable qualitative analysis without overwhelming computational or human resources. Demonstrating that a dataset is biased does not require examining every instance: if a bias appears in a random sample, it is a flaw of the creation process. The AIETs' questions acted as provocateurs, evoking different perspectives and analysis methods for the same dataset, but the analysis and its results depended fundamentally on the human factor. We hope these findings contribute to urgent public discussions about ethics, transparency, accountability, and AI regulation by offering a practical pathway for responsibly auditing massive datasets.

0.31

An Open Pipeline and Dashboard for Systemic-Risk Evidence under the EU AI Act's Code of Practice

cs.AI ·2026-09-23 ·Jacob T. Emmerson, Phuong-Anh Nguyen-Le, Ronan Romano, Wilber Sean V. Anterola, Yann Billeter +1 more
surveybenchmark similarity 0.59
Abstract · 6 pages, 5 figures, submitted to EACL 2027 Systems Demonstration track

Claims about AI safety reach audiences well beyond the AI community, yet many rely on opaque evidence or static assessments, when supporting evidence is accessible at all. We present the Systemic Risk Index, an open evaluation pipeline and dashboard built to make empirical evidence more transparent and traceable to the public. Our work organizes 19 public benchmarks into four systemic-risk categories defined by the EU GPAI Code of Practice---CBRN, cyber offense, harmful manipulation, and loss of control---and evaluates models using harm-preserving perturbations and simulated deployment contexts. The interactive dashboard lets users alternate between average and worst-case aggregation, vary how model capability affects the aggregate score, and trace each risk rating to its benchmark evidence. Across 18 models, scores fall by 14 to 37 points under worst-case aggregation, highlighting information that can be hidden by an average assessment of model risk. LLM judges show agreement with human graders comparable to human--human agreement ($κ= 0.78\text{--}0.82$), and a blind audit finds that $83\%$ of sampled transformations preserve the original harm. In a survey ($N = 21$), most participants report that scores are easy to understand and that the dashboard encouraged them to view model evaluations under different settings

0.14

Safe by Benchmark, Unsafe in Practice: Lessons from 1331 Child-Related AI Incidents

cs.AI cs.CY ·2026-10-26 ·Junfeng Jiao, Kevin Chen, Shrestha Mishra, Abhejay Murali, David Atkinson +2 more
benchmarkcode/data similarity 0.58
Abstract · 17 pages, 10 figures. Dataset and documentation: https://github.com/The-Responsible-AI-Initiative/KidAID

AI systems are increasingly present in children's everyday environments: education, entertainment, social networks, and toys. Their safety is typically assessed at the model level, through benchmarks, red-teaming, and content filtering. However, most AI harm taxonomies lack child-specific evidence and under-specify developmental harm and the stakeholder asymmetries unique to minors. Building on prior taxonomies, we construct a child-centered harm taxonomy mapped to sociotechnical failure modes. We curate and code 1331 publicly reported child-related AI incidents, annotating internal and external harm types and the failure mode implicated in each incident. Our analysis reveals which harms are most prevalent in public reporting, how individual-level and population-level harms relate, and which failures are most often coded alongside harm. We find that harm concentrates in representational, dignity-related, and psychological categories, and co-occurs most often with deployment-context, governance, and misuse failures rather than model defects. Harm to an individual child is also frequently linked to population-level harm, reaching children who never interacted with the system. These findings suggest that model-level evaluations, while necessary, cannot by themselves anticipate the harm pathways most often documented for children. We conclude with actionable implications emphasizing mitigations beyond model-level safeguards such as content filtering: age assurance, consent verification for depiction, safety-by-design UX constraints, incident disclosure norms, and auditing regimes targeting developmental risk. Privacy-minimized annotations, a codebook, and documentation are available at https://github.com/The-Responsible-AI-Initiative/KidAID.

0.39

"I Thought You Were The Uncensored Place": Norms, Rules, and Moderation in AI-Generated Sexual Content Communities

cs.CR ·2026-08-13 ·Lucy Qin, Jaron Mink, Elissa M. Redmiles
published similarity 0.58
Abstract · To appear at the AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES) 2026

As AI-generated sexual content (AIG-SC) is increasingly produced, online communities have emerged to support creators' needs. To understand whether and how community governance attempts work to prevent abuse while supporting free expression, we interviewed 24 members and moderators of large AIG-SC online communities (10,000+ members) with stated rules against creating and sharing abusive content (e.g., AI-generated CSAM). Through in-depth interviews, we offer insight into: (1) how and why these communities form; (2) implicit community norms; (3) explicitly stated rules---and their operationalization via content moderation; and (4) tensions between community values and moderation that leave space for abusive behavior. Our findings reveal a complex picture: while many creators and communities have personal boundaries against abuse, advice and resources for creating any form of AI-generated sexual content are accessible to users regardless of their intentions. Further complicating community moderation are norms that center anti-censorship and non-judgment, which leave moderators to justify their actions using the limits of the law and terms of service. We end by reflecting on the ways in which technical, community, and legal governance may most effectively mitigate the production of abusive content.

0.50