CS Paper Radar

The AI Risk Observatory: What Can We Learn from AI Disclosures in Annual Reports About Societal Resilience?

Bart Jaworski, Andrew Sutton

cs.AI ·submitted 2026-10-01 12:03 UTC · v2
code/data

Abstract

Societal resilience research relies on access to useful and actionable data, which motivates our main research question: Can annual reports, processed at scale with LLMs, provide a useful signal about how companies disclose their response to AI? We test this by applying a reproducible two-stage classification pipeline to 9,821 annual reports from 1,362 UK listed companies (2020-2025, with partial 2026 data). We first validate the method against 474 human-annotated passages, finding high recall and moderate label-level agreement. We then report three empirical patterns: (i) between 2020 and 2025, the share of reports mentioning AI risk rose from 2.8% to 41.2%, while AI adoption disclosure also rose, from 13.8% to 45.2%, and named vendor mentions cluster around a small set of major providers led by Microsoft; (ii) disclosure varies substantially by Critical National Infrastructure sector and market segment: AIM reports disclose AI risk at far lower rates than Main Market reports, and sectors such as Energy and Data Infrastructure lag behind the rest in AI risk disclosure; and (iii) harm disclosures are near-absent (seven reports across the entire corpus). We develop a substantiveness classification to assess the quality of the disclosure and find that most AI risk disclosure is not substantive: in 2025, 41.2% of all reports mention AI as a risk, but only 4.3% contain AI risk disclosure we classify as substantive.

Comment
22 pages (9 main text + appendices), 12 figures, 7 tables. Code and data: https://github.com/84rt/AI-Risk-Observatory (release dataset-v1.1)

Closest papers

Nearest neighbours by meaning across the whole index.

Unmitigated? How are the world's largest companies responding to AI risks?

cs.CY ·2026-11-03 ·Alexander K. Saeri, Jess Graham, Michael Noetel, Peter Slattery, Kun Zhao +3 more
code/data similarity 0.72
Abstract · Access data and code at https://osf.io/76z2f/

AI is introducing new risks to large companies and amplifying existing risks, but whether companies are addressing the AI risks they are exposed to is largely untested. We investigated what outsiders can observe: whether companies publicly acknowledge AI risks. We coded public documents from 200 of the largest companies worldwide, collected in early 2026, against the 24 AI risks in the MIT AI Risk Repository, and compared acknowledgements with expert judgements of risk severity, sector vulnerability, and AI role responsibility from a companion Delphi study of 272 AI risk experts. Of the 100 largest companies, 32% did not publicly acknowledge any AI risks. Of the largest 1000 companies, we estimate that 62% [95% CI: 54%, 70%] did not publicly acknowledge any AI risks. Where companies did acknowledge risks, five accounted for over half of all acknowledgements: discrimination, weapons & cyberattacks, fraud & scams, loss of privacy, and devaluation of human creativity. Several risks that experts judged most severe were almost absent: almost no company acknowledged power centralization (concentration of power among those who control advanced AI), competitive dynamics (racing to deploy AI at the expense of safety), or dangerous capabilities. Companies in sectors experts judged more vulnerable to a risk were more likely to acknowledge that risk. But even companies in sectors judged 'extremely vulnerable' to a risk usually did not acknowledge it. Companies with roles experts judged most responsible for addressing a risk were no more likely to acknowledge it than other companies. General-purpose AI developers acknowledged AI misalignment (15%) far less often than discrimination (85%). These findings provide an empirical baseline for the design of AI risk disclosure requirements.

0.28

Mapping U.S. Federal AI Governance Against Sector Vulnerability

cs.CY cs.AI ·2026-09-14 ·Ho Ting Hung, Angelica Chowdhury, James Teague, Simon Mylius, Spencer Michaels +3 more
similarity 0.66
Abstract

Artificial intelligence (AI) poses different levels of risk across sectors, but are these differences reflected in U.S. federal AI governance? To help answer this question, we assess 684 federal AI governance documents for their coverage of 14 sectors and 24 AI risks. We measure coverage as breadth (i.e., how frequently the risk or sector is addressed across documents) and depth (i.e., how substantively the risk or sector is discussed). We then compare sector coverage patterns for each of the 24 risks with vulnerability assessments from a Delphi study of 272 experts. Our analysis finds substantial variation in coverage: AI risks related to robustness, system security, and governance receive more attention than socioeconomic, environmental, and emerging risks, including multi-agent risks. Public administration, national security, information, and scientific services receive comparatively high levels of coverage relative to other sectors, such as finance and healthcare, which experts rate as highly vulnerable to AI risks. By mapping current coverage and identifying where it differs from expert assessments of vulnerability, we surface potential AI governance gaps which may help inform AI risk-related decisions across government and industry.

0.06

Beyond Predictable Paths: Redefining AI Security Incident Reporting for Agents

cs.CR cs.AI ·2026-09-21 ·Anastasia Pustozerova, Eugene Bagdasarian, Luca Beurer-Kellner, Battista Biggio, Nico Ebert +20 more
similarity 0.65
Abstract · under submission, mega paper (authorship does not imply endorsement of every sub-section)

AI agents are being deployed rapidly, accompanied by a growing number of AI-specific attacks and corresponding incidents. As incident reporting becomes increasingly important for legal compliance, governance, accountability, and security; current frameworks must be adapted to the unique characteristics of AI agents. In this paper, two editorial authors compare AI systems and AI agents and, drawing on input from 23 experts in academia and industry, identify the information required for reporting incidents where the security of AI agents is harmed. %involving AI agents. Potential reporting elements include, for example, agent memory and memory accesses, actual and potential levels of autonomy, and tool usage. Based on these findings, we identify several open research questions, including how to efficiently record incidents and how to determine whether vulnerabilities and incidents generalize. Expert feedback also highlighted potential reporting weaknesses, such as risks of data leakage and attacks targeting the reporting infrastructure itself, creating additional research needs. Lastly, we summarize privacy requirements and outline research directions for the secure and trustworthy deployment of AI agents.

0.18

Drowning in AI Slop: How Social Media Platforms (Do Not) Label AI and Deepfake Content under EU law

cs.CY ·2026-09-29 ·Bram Rijsbosch, Luka Bekavac, Henry Tari, Gijs van Dijck, Konrad Kollnig
similarity 0.64
Abstract · This work is under submission and has not undergone peer review yet

AI labels are emerging as a primary safeguard for transparency about AI-generated content on social media, including under the EU Digital Services Act and AI Act. Yet, limited systematic evidence exists on how platforms implement such labels in practice. We conduct a legally grounded audit of AI labelling across Instagram, TikTok, X, and YouTube, drawing on the European Commission's July 2026 guidelines on deepfakes. To this end, we analyse platform policies and detection approaches, 10,722 posts collected via systemic-risk and AI-related keywords, an expert-annotated subset of 500 posts, and controlled uploads to the four platforms of outputs from ten popular generative AI tools. We find that AI labelling is now broadly established. All four platforms apply labels automatically and a greater share of labels are platform-applied than in earlier audits. However, coverage remains incomplete where labelling arguably matters most: only 33% of expert-identified deepfakes in systemic risk contexts carried a platform-applied AI label, while reaching a median of 160,000 views. In controlled uploads of AI-generated content carrying standard AI provenance signals, platforms labelled only 61% of uploads, and commonly strip those signals after uploading. Overall, platform rules, label designs, detection approaches, and reporting diverge substantially. In response, we identify concrete opportunities to improve the uptake, clarity and efficacy of AI labelling.

0.05

Expectations and Practices around AI Disclosure in CS Research

cs.CY cs.CLcs.HC ·2026-08-24 ·Arati Mohapatra, Danish Pruthi
top venuesurvey similarity 0.64
Abstract · Accepted to EMNLP 2026 (Findings)

As generative AI tools find increasing use in research workflows, ongoing debates on their impact, appropriateness and responsible use have led policymakers to enact policies to disclose AI use at multiple publishing venues. However, are current AI disclosure policies and practices reflective of their purpose? In this work, we first investigate disclosure policies of top computer science venues and find that despite their prevalence, they remain highly under-specified. Secondly, through a survey of computer science researchers (N=$109$), we characterize the necessity of disclosures across different research tasks and levels of human involvement. We learn that researchers find disclosures most necessary for tasks involving research design, and for tasks when the human involvement is low. We also compile expectations that researchers have about the information to be conveyed in AI disclosure statements. Lastly, through an analysis of $13867$ disclosure statements from EMNLP $2025$ and ICLR $2026$, we reveal a large disconnect between these expectations and AI disclosures in practice---a prime example being writing assistance which is deemed less necessary but is frequently disclosed. We conclude with recommendations to align AI disclosure policies and practices with expectations, suggesting a categorization of research tasks by perceived necessity and a boilerplate template capturing expected details.

0.46

An Open Pipeline and Dashboard for Systemic-Risk Evidence under the EU AI Act's Code of Practice

cs.AI ·2026-09-23 ·Jacob T. Emmerson, Phuong-Anh Nguyen-Le, Ronan Romano, Wilber Sean V. Anterola, Yann Billeter +1 more
surveybenchmark similarity 0.63
Abstract · 6 pages, 5 figures, submitted to EACL 2027 Systems Demonstration track

Claims about AI safety reach audiences well beyond the AI community, yet many rely on opaque evidence or static assessments, when supporting evidence is accessible at all. We present the Systemic Risk Index, an open evaluation pipeline and dashboard built to make empirical evidence more transparent and traceable to the public. Our work organizes 19 public benchmarks into four systemic-risk categories defined by the EU GPAI Code of Practice---CBRN, cyber offense, harmful manipulation, and loss of control---and evaluates models using harm-preserving perturbations and simulated deployment contexts. The interactive dashboard lets users alternate between average and worst-case aggregation, vary how model capability affects the aggregate score, and trace each risk rating to its benchmark evidence. Across 18 models, scores fall by 14 to 37 points under worst-case aggregation, highlighting information that can be hidden by an average assessment of model risk. LLM judges show agreement with human graders comparable to human--human agreement ($κ= 0.78\text{--}0.82$), and a blind audit finds that $83\%$ of sampled transformations preserve the original harm. In a survey ($N = 21$), most participants report that scores are easy to understand and that the dashboard encouraged them to view model evaluations under different settings

0.14

Transparency Disclosures for Chain-of-Thought Monitorability

cs.CY cs.AI ·2026-11-10 ·Jacob Davies, Theo Bearman
position similarity 0.62
Abstract · 27 pages (including references and appendix), 3 figures

Chain-of-thought (CoT) monitoring is one of the most valuable tools for monitoring advanced AI systems, but it is under threat. We argue that frontier AI developers should report specific indicators to make these threats trackable and to better facilitate coordination to preserve CoT monitorability. We identify five indicators that are both highly relevant and reasonably feasible to disclose, one for each link in a five-step chain on which CoT monitoring depends. The first, third-party CoT access and base architecture specifications, also underpins external verification of the rest. The others span CoT quality (naturalness and faithfulness) and monitoring effectiveness (detection and uplift). An audit of current disclosures by Anthropic, OpenAI, and Google DeepMind shows that no developer covers the full set, but that every indicator has been reported by at least one frontier developer. Because current disclosures rely largely on developers' own evaluations, we also recommend that a cross-developer audit be repeated with deeper access than public disclosures allow, either by embedded evaluators working within developers or by an Independent Verification Organization.

0.30

Assessing Company Contributions to Societal Resilience: Extending the Societal Capacity Assessment Framework to Agentic AI

cs.CY ·2026-08-27 ·Catherine Simons, Alexander K. Saeri, Peter Slattery, Neil Thompson
similarity 0.61
Abstract · 19 Pages

Companies that deploy AI agents and make them available to others are creating the sociotechnical circumstances under which this technology integrates into existing social and economic structures. AI-deploying companies are institutional actors that actively shape society's capacity to withstand and govern the consequences of agentic AI. In view of these societal impacts, companies can build societal resilience by designing and promoting safer implementations of AI agents. To operationalize this goal, this paper adapts the indicator-based Societal Capacity Assessment Framework (SCAF) to measure how a company's deployment decisions contribute to societal resilience, inverting its original measurement of societal resilience as a backdrop for deployment decisions (Gandhi et al., 2025). Our procedure has two steps: a conceptual step in which we design a suite of indicators that define what SCAF's vulnerability, coping, and adaptive capacities mean when assessing a company's agentic AI deployment decisions; and a measurement step in which we apply this framework in a structured assessment of public-facing Microsoft documents.

0.29