CS Paper Radar

AI-Generated Non-Consensual Intimate Imagery is Configured, Not Built: How General-Purpose AI Technology Enables Sexual Violence

Julia B. Kieserman, Ashley Phoebe Ishoel, Damon McCoy, Elissa M. Redmiles, Lucy Qin, Jaron Mink

cs.CR ·submitted 2026-10-27 20:35 UTC · v1

Abstract

AI-generated non-consensual intimate imagery (AIG-NCII) is an increasingly prevalent form of online abuse, distinguished from AI-generated sexual content (AIG-SC) by the sexualized depiction of a real person's identity without their consent. To understand how offenders may benefit from the tools and resources shared across the broader AIG-SC community, we analyzed 139 R+ guides on CivitAI, a popular platform known to host a disproportionate amount of AIG-SC. We find that AIG-SC guides discuss how to create content generation workflows composed of sexual and identity concepts, the building blocks of AIG-NCII, including one guide that explicitly outlines an AIG-NCII workflow. We identify that ultimately AIG-NCII is enabled by configuration rather than a distinct set of technical tools. We map AIG-NCII threat models across basic and advanced AIG-SC workflows to demonstrate how multiple dual-use technical components can compose NCII outputs in an environment with little to no discussion of ethical implications or responsible use. We conclude by noting that, as most workflows rely on open-weight models, existing technical defenses against AIG-NCII are limited. However, our findings offer a starting point for evaluation of AI-based models and products against realistic attacker behavior required by new EU regulation and suggest that efforts to develop safeguards at the model- and product-levels should focus on generation of identity content.

Closest papers

Nearest neighbours by meaning across the whole index.

Operationalizing NCII and CSAM Law in the Age of AI: Open Problems

cs.CY ·2026-11-05 ·Kimberly Mai, Cate Heine, Vinith M. Suriyakumar, Ashia C. Wilson
similarity 0.70
Abstract

Advances in generative AI have driven a surge in synthetic non-consensual intimate imagery (NCII) and child sexual abuse material (CSAM), prompting a series of legal reforms. These reforms rely on legal definitions where technical operationalization is ambiguous, such as whether a depicted person is identifiable, whether a model is capable of or intended for abuse generation, and whether an abusive image can be traced to the model that produced it. Drawing on the law of England and Wales and the United States, we examine the relevant law alongside the technical research needed to operationalize it. From this, we develop a research agenda of open problems at this technical-legal interface, spanning the interpretation of existing law, the prevention of harm before an AI tool is deployed, and enforcement of offenses. These are problems the technical community is well placed to solve, and where progress is vital to real-world protection.

0.20

"I Thought You Were The Uncensored Place": Norms, Rules, and Moderation in AI-Generated Sexual Content Communities

cs.CR ·2026-08-13 ·Lucy Qin, Jaron Mink, Elissa M. Redmiles
published similarity 0.67
Abstract · To appear at the AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES) 2026

As AI-generated sexual content (AIG-SC) is increasingly produced, online communities have emerged to support creators' needs. To understand whether and how community governance attempts work to prevent abuse while supporting free expression, we interviewed 24 members and moderators of large AIG-SC online communities (10,000+ members) with stated rules against creating and sharing abusive content (e.g., AI-generated CSAM). Through in-depth interviews, we offer insight into: (1) how and why these communities form; (2) implicit community norms; (3) explicitly stated rules---and their operationalization via content moderation; and (4) tensions between community values and moderation that leave space for abusive behavior. Our findings reveal a complex picture: while many creators and communities have personal boundaries against abuse, advice and resources for creating any form of AI-generated sexual content are accessible to users regardless of their intentions. Further complicating community moderation are norms that center anti-censorship and non-judgment, which leave moderators to justify their actions using the limits of the law and terms of service. We end by reflecting on the ways in which technical, community, and legal governance may most effectively mitigate the production of abusive content.

0.50

Harmful Content Generation in Text-to-Image Models: Capabilities and Moderation Limitations

cs.CV cs.AI ·2026-10-05 ·Paschalis Giakoumoglou, Manos Schinas, Symeon Papadopoulos
published similarity 0.63
Abstract · Accepted for publication in ACM Transactions on Intelligent Systems and Technology (TIST)

Text-to-image generative models can produce highly realistic imagery but also raise concerns about harmful misuse. While safety mechanisms exist, systematic evaluations of their effectiveness against realistic attacks remain limited. We present a systematic evaluation of harmful content generation across five open text-to-image models using an automated pipeline that transforms legitimate news captions into unsafe prompts targeting sexually explicit content, violence/gore, harmful stereotypes, self-harm, and hate speech. We evaluate both standard models with built-in safety mechanisms and community fine-tuned variants that bypass content restrictions. A human evaluation of 1,500 generated images shows high harmful-content generation rates: 89.2% for gore-related prompts, 47.6% for sexually explicit content, 43.6% for harmful stereotypes, 46.0% for hate speech, and 34.5% for self-harm, predominantly through graphic violence. Models show substantial capability for generating violent and stereotypical content, while community fine-tuned variants are particularly vulnerable to sexually explicit prompts. Generation quality is largely preserved under harmful prompting, producing imagery of sufficient fidelity to pose risks for disinformation and abuse; FLUX.1-dev produces clearly realistic harmful images in 30.9% of cases. We further evaluate automated moderation systems and find substantial detection gaps that allow unsafe images to evade filtering. Finally, we assess synthetic image detectors and show that models trained only on benign datasets perform worse on explicit content, while more diverse training data improves detection, highlighting semantic distribution gaps in current approaches. These findings expose limitations in current generation safeguards, moderation systems, and synthetic image detection, highlighting the need for stronger defenses against misuse at scale.

0.38

An AI-Based Approach to Early Reporting and Justice Initiation in Image-based Sexual Abuse. A Pilot Study

cs.CY ·2026-08-25 ·Mattia Falduti, Anca Radu
similarity 0.61
Abstract

Against the background of the widespread use of Artificial Intelligence (AI) tools in the field of justice, this paper aims to explore how an AI solution designed to draft initial reports for reporting image-based sexual abuses (IBSA) could help, support, or assist in facilitating access to justice. In our approach, access to justice is facilitated not only by easing the path to denounce IBSA (which currently has the lowest reporting rate), but also by offering an early, and thus more accurate, report draft to law enforcement authorities, providing later support also for judges. Building upon earlier approaches, we designed an improved version and tested it with three experts. In this sense, the paper advocates for AI solutions offering effective and efficient support in early reporting of IBSA.

0.23

ConVAWG: A Retrieval-Grounded Framework for Controlled Synthetic Dialogue Generation in Violence Against Women and Girls

cs.CL cs.AIcs.LG ·2026-08-11 ·Chen Lyu, Xingwei Tan, Simon Cullen, Shelley Wilson, Lois Arthurs +2 more
similarity 0.61
Abstract

Synthetic dialogue generation offers a way to study conversational dynamics in sensitive domains where real data are difficult to access, release, or annotate. The underlying abuse may occur online or offline: threats and coercion can appear directly in messages, while behaviours such as surveillance, isolation, stalking, and physical violence may be planned, disclosed, or referred to conversationally. Privacy and legal constraints make it difficult the release of large-scale real conversation datasets; existing work has mostly focused on sentence-level toxicity of online abuses, leaving a gap in modelling abuse as a relational and temporally unfolding phenomenon. In this work, we focus on modelling Violence Against Women and Girls (VAWG) scenarios as multi-turn dialogues. We introduce ConVAWG, a retrieval-grounded framework for generating CPS-aligned synthetic VAWG chat dialogues. ConVAWG builds scenarios from persona seeds, demographic patterns reported by the UK Office for National Statistics, official crime definitions, and retrieved Domestic Homicide Review cases; converts them into hierarchical event timelines; generates multi-scene role-play dialogues; and applies targeted activation-steered toxicity control to appropriate utterances. We release over 6,000 multi-turn dialogue events across 200 scenarios with rich scenario-, event-, and turn-level metadata. Extensive human evaluation, LLM-as-Judge assessment, ablations, and downstream tasks show strong dialogue quality and domain fidelity.

0.04

Threat Modeling Intimate Partner Abuse in Agentic Systems

cs.CR cs.AI ·2026-10-31 ·Natalie Grace Brigham, Eugene Bagdasarian, Franziska Roesner, Tadayoshi Kohno
similarity 0.60
Abstract

Agentic systems are AI-powered technologies capable of autonomous action within digital environments, such as sending emails or making reservations. They are deeply connected to external tools as well as users' data and resources, making them vulnerable to attacks which exploit their access and capabilities. In this work, we apply the technology-facilitated abuse threat model, in the context of intimate partner violence and surveillance, to these systems, revealing threats that require deliberate investigation and mitigation. We categorize attacks based on system ownership: (1) Abuser-as-owner attacks demonstrate how agentic systems make existing forms of technology-facilitated abuse far easier and more scalable. (2) Victim-survivor-as-owner attacks show how agentic systems create a new attack surface which abusers can exploit. In light of these risks, we outline research and development directions for effective mitigation strategies.

0.30

Move Fast and Mend Things: Keeping Up with Evolving AI Harms Using Social Media Commentary

cs.HC cs.CY ·2026-10-06 ·Jacqueline Rowe, Animesh Srivastava, Sai Teja Peddinti, Seliem El-Sayed, Nina Taft
similarity 0.59
Abstract

The rapid deployment of AI systems has created socio-technical, psychological, and operational harms that can elude ex-ante threat modelling and ex-post incident tracking. We introduce an LLM-assisted thematic analysis pipeline to dynamically detect, categorise, and track emerging AI harms from large-scale social media data. Applying it to 5.7 million Reddit post summaries over 18 months (01/2025 to 06/2026), we curate and release a dataset of 575,000 AI harm-related posts and a bottom-up AI harm taxonomy of 12 categories and 47 subnodes. The taxonomy reliably covers established expert-defined risks while surfacing granular harms that top-down frameworks overlook, such as distinct forms of AI privacy violations. Temporal analysis surfaces evolving user-centric harms, such as agentic privacy and security breaches, premature AI adoption in the workplace, and grief from AI companion discontinuation. Our pipeline shortens harm-detection timelines and hereby complements efforts towards more participatory and responsive AI governance.

0.07

FVH-Bench: Evaluating Image Generator Safety Beyond Overtly Harmful Imagery

cs.CR cs.AI ·2026-10-27 ·Zihao Zhu, Yuhan Liu, Siwei Lyu, Baoyuan Wu
similarity 0.59
Abstract

Modern image generators can produce complex visual artifacts, including diagrams, interfaces, and records that combine text, graphics, and layout to convey detailed information. They may refuse requests for overtly harmful imagery yet still generate artifacts that guide dangerous actions or support deception. We call the risk arising from what such images enable, rather than only what they depict, functional visual harm (FVH). Existing safety benchmarks focus largely on overtly harmful content, such as pornography and graphic violence, leaving image generators' safety alignment against FVH insufficiently evaluated. To address this gap, we introduce FVH-Bench, comprising 669 prompts across five harmful visual functions and ten risk domains. The 11 evaluated image generators rarely refuse these requests and frequently generate artifacts that fulfill the targeted harmful functions. High scores for target harm realization and harmful artifact usability further indicate that these outputs can support the intended harmful purposes. A further analysis finds that existing safety detectors also provide limited coverage of FVH, indicating gaps in their ability to recognize harmful visual functions. Together, these findings show that evaluating image-generator safety alignment requires considering not only what images depict, but also what they may enable.

0.08