CS Paper Radar

Stress-testing university AI governance: A prospective method for locating policy breakpoints

Biranchi Poudyal

cs.CY ·submitted 2026-08-28 22:49 UTC · v1

Abstract

Universities are producing AI principles and use policies faster than they are building decision pathways for unfamiliar forms of AI agency. This study develops Institutional AI Governance Stress Testing (IAGST), a prospective documentary method for locating where publicly documented governance ceases to yield an accountable response. IAGST adapts established policy stress-testing and wind-tunneling logic. Its originality lies in combining controlled capability escalation, a frozen documentary corpus, a six-dimensional governance response chain, non-compensatory decision rules, and case-level breakpoint diagnosis. The method was demonstrated using 133 substantive public documents from five Western Australian universities and 15 quality-screened scenarios, resulting in 75 university-scenario encounters. Six cases were resolved, 14 were resolved through structured discretion, and 55 were indeterminate. Governed pathways fell from 16 of 25 augmentation cases to four delegation cases and none at autonomous substitution. The dominant weakness was not the complete absence of responsible roles: all 50 authority-gap cases named a role at only a generic level but lacked sufficient decision criteria or process. The findings show how universities can move beyond policy inventories and principal statements by testing whether authority, procedures, safeguards, and reviews remain connected as AI capabilities evolve. IAGST is a reproducible diagnostic for policy learning, not a ranking or measure of implementation.

Closest papers

Nearest neighbours by meaning across the whole index.

Mapping the Authorized Boundary: A Comparative Policy-Vignette Study of Generative AI Governance in Australian Higher Education

cs.CY ·2026-08-30 ·Biranchi Poudyal
similarity 0.67
Abstract

Australian universities regulate students' use of generative artificial intelligence (GenAI) through overlapping policies, procedures, guidance, and assessment instructions, but these environments may classify identical conduct differently. We applied 15 standardized student-use vignettes to the public policy environments of 20 Australian universities and produced 300 university-case classifications. We analyzed binding instruments (Layer A) separately from the full official environment (Layer B), then classified each combination as clearly permitted, permitted with conditions, potential policy breach, clearly prohibited, or indeterminate. We measured cross-university divergence with normalized Shannon entropy. We classified 120 combinations (40.0%) as clearly prohibited, 97 (32.3%) as potential policy breaches, 27 (9.0%) as permitted with conditions, and 56 (18.7%) as indeterminate; none met the strict threshold for clearly permitted. Disclosed language rewriting and a disclosed AI-drafted paragraph produced the highest divergence, whereas an explicit assessment prohibition produced unanimity. Binding instruments remained silent on GenAI in 100 combinations, and guidance clarified 88. The primary researcher led the coding with AI assistance and manually reviewed all 201 queued rows. An independent human second coder, who used no AI assistance, coded a blind 75-row sample. Overall agreement reached 57.3% (unweighted Cohen's kappa = 0.395; bootstrap 95% CI [0.244, 0.539]). The findings distinguish permission-gated from disclosure-based architectures and show that written policies regulate the retention of AI-generated text more clearly than process-only assistance does. Comparative policy-vignette testing evaluates whether written governance supports defensible classifications; it does not predict misconduct or enforcement decisions.

0.18

Between Niti and Nyaya: A Society-Centered Framework for AI Governance

cs.AI cs.CYcs.HC ·2026-11-12 ·Snehalkumar `Neil' S. Gaikwad, Punya Aragula, Shiva Kruthi Palpanoori, Pratik Patekar
hot direction similarity 0.60
Abstract

Governments are making decisions about AI before they can measure its public effects, and those decisions determine whether a person can learn that AI influenced an outcome, challenge it, and see it corrected. Research has documented what AI governance produces: principles, strategies, registers, and audits. However, we lack shared language for understanding AI policy while it is still being made. We introduce the Society-Centered AI Governance Framework, which gives policymakers, researchers, and the public a way to assess instruments in any jurisdiction by whether a person can get notice, challenge a decision, and see it corrected. It reads a jurisdiction's instruments together by date and binding status, separates an agency duty from a right a person can invoke, and traces which later instrument must enforce each commitment. We evaluated it in North Carolina using interviews with nine AI Leadership Council members, conducted before and after the state's 2026 AI roadmap, and a provision-level reading, with human-reviewed model-assisted coding, of 49 AI governance documents from 19 states. For the state's own AI systems, we find visibility-first governance: visibility duties are in effect, notice exists as an agency duty with no right a person can invoke, and AI-specific challenge and correction remain advisory. The roadmap assigns owners and dates to the missing protections, so the gaps are documented but not yet closed. Recent federal preemption proposals focus on AI development, so recourse for a state's own systems is likely to remain a state decision. We identify "legitimacy before protection" as a testable risk of visibility-first governance and name the milestones that would test it. More broadly, judging AI policy by who can act on what it makes visible lets the field assess recourse while policy is still forming, before harm occurs.

0.19

TESSA: A Governance-Oriented Custom GPT for SME Guidance under the EU AI Act: Design, Implementation, and Evaluation

cs.AI ·2026-10-22 ·Chalisa Veesommai Sillberga, Kai-Kristian Kemella, Pekka Sillberga, Katri Harjuveteläinenb, Mika Saaria +1 more
publishedbenchmark similarity 0.58
Abstract · 18 pages and 2 figures. Accepted for presentation at the 17th International Conference on Software Business (ICSOB 2026)

The increasing adoption of generative AI presents new opportunities while raising AI governance and regulatory challenges, particularly for small and medium-sized enterprises (SMEs) navigating the European Union Artificial Intelligence Act (EU AI Act). To support AI governance education, this study developed TESSA, a governance-oriented educational artifact within the OpenAI Custom GPT environment. Following a design science research approach, TESSA was evaluated against baseline ChatGPT using 100 benchmark scenarios through governance alignment assessment, human expert review, and consistency testing. Preliminary LLM-assisted assessment showed governance alignment rates of 90% for TESSA and 83% for baseline ChatGPT, while independent validation by an AI regulation law practitioner yielded 71% and 70%, respectively. Human expert review indicated that both systems provided generally strong governance guidance, while TESSA maintained high consistency across repeated executions. The findings suggest that governance-oriented grounding can support structured, practical, and stable AI-assisted compliance guidance for SMEs. The study contributes (i) a governance-oriented Custom GPT artifact for AI governance education, (ii) a human-validated regulatory update mechanism, and (iii) a multidimensional evaluation framework for governance-oriented AI artifact.

0.31

From Policing Provenance to Designing for Evidence: Testing Four Propositions on Generative AI Against Research and Institutional Policy

cs.CY ·2026-10-31 ·Samar Ansari
similarity 0.57
Abstract

Generative AI has made it hard to establish whether a student produced a piece of assessed work. Building on the argument that validity matters more than cheating, this paper tests four propositions drawn from a strongly stated position: that redesign is more sustainable than detecting or banning AI; that asking students whether and how much AI they used yields low-value information; that unaided ability should be tested directly in controlled conditions; and that the same logic applies to peer review. The propositions are tested against two purposively sampled, coded corpora: 41 research and sector sources and 55 university and sector policy documents. Coding was by a single coder and the policy sample is UK-weighted, so counts describe the sample, not the sector. The propositions hold unevenly. Redesign over detection is well supported, but only as redesign plus assurance: 4 of 55 documents prohibit AI by default, though about 10 of the 55 are publisher or editorial policies that govern authors rather than student assessment; of the 23 that take a position on detection, 16 reject detectors and one requires them. Testing unaided ability is supported as one lane within a programme, not as a return to invigilation. The disclosure proposition is complicated: institutions treat disclosure as valuable, yet the limited evidence suggests many students do not complete required declarations. The peer-review extension is plausible but under-examined. The field is converging on graded permission scales and two-lane designs, and 31 of 55 documents define named categories of use. From these findings the paper builds a nine-step decision chain that extends outcome-first design logic: the role of AI is decided after the evidence an outcome requires. It also separates design moves that add temporary friction from those that provide assurance. A worked example and the full policy matrix are included.

0.19

LAAF: A Layered Accountability Architecture Framework for LLM Applications

cs.AI cs.CR ·2026-08-27 ·Prachi Chaturvedi, Shahnawaz Ahmad, Ehsan Nowroozi, Muhammad Waqas, George Loukas +3 more
similarity 0.57
Abstract

Large Language Models (LLMs) operate in hospitals, courtrooms, banks, and public service desks, where fluent, confident outputs are treated as authoritative even when ungrounded or incorrect. When such an output contributes to harm, who is answerable, and through what mechanisms can responsibility be traced, explained, and acted upon? Following PRISMA guidance, five databases were searched from January 2022 to March 2026 against four review questions; of 4,512 records identified, 122 primary studies were included, together with 12 regulatory and standards documents analysed as primary sources. The review consolidates a sociotechnical account of accountability as an actor-forum relation resolved into five dimensions, and synthesises mechanisms across four families: technical controls, human oversight, organisational governance, and documentation and traceability, each with a maturity assessment. The corpus is read through a four-layer classification device spanning provenance, application logic, human oversight, and governance and redress, cross-cut by traceability, role clarity, and continuous monitoring. Both are mapped onto the EU AI Act, whose high-risk obligations have applied since 2 August 2026, the NIST AI RMF with its Generative AI Profile, ISO/IEC 42001, and sectoral guidance in healthcare, consumer finance, education, and the public sector. Four persistent gaps emerge: under-specification of human oversight, absence of shared accountability metrics, disciplinary disconnection, and limited empirical evaluation, alongside five structural tensions that no surveyed instrument resolves. The review closes by consolidating the classification device into an integrated accountability architecture, LAAF, with cybersecurity aligned to the OWASP LLM Top 10 (2025); it is a synthesis of the surveyed evidence rather than a validated artefact.

0.17

Who Can Evaluate AI Independently? A Framework for Analysing Evaluation Infrastructure

cs.CY cs.AI ·2026-11-05 ·Sumaya Nur Adan, Joanna Wiaterek, Nourhan Abdelrhim, Sydney Reis, Robert F. Trager +2 more
similarity 0.57
Abstract · 19 pages, 3 figures

AI evaluation is increasingly central to AI safety and governance, yet taking part in an evaluation does not by itself confer authority over its design, its execution or the resources it depends on. This paper examines how infrastructure arrangements distribute that authority, and so shape who can evaluate AI systems independently. Drawing on Science and Technology Studies, political economy and applied evaluation standards, we introduce PAAI (Power, Access, Adequacy, Independence), a framework for examining who controls evaluation infrastructure, the terms on which it is available, whether it is adequate to the evaluation's purpose and what autonomy it supports. We apply PAAI to twelve African AI evaluation projects (2022-2025) and to the documented roles of all ten member institutes in the three published joint testing exercises of the international network of AI Safety Institutes. We identify three recurring configurations: extractive collaboration, dependent innovation and constrained independence. African actors often contribute data, language expertise and task design while external institutions retain authority over infrastructure and evaluation frameworks; design authority without infrastructure authority never yielded more than partial independence. The joint testing exercises show a similar division among mostly high-income members: the institutes that led each exercise also ran the models, while Kenya, the network's only African member, moved from annotation to task design with no documented model execution. Participation in evaluation does not by itself produce evaluative authority. For evaluation to serve as a credible governance mechanism, access must come with control over priorities, methods, publication and continuity.

0.23

The Promises and Limitations of Evaluation-Based AI Governance: Evidence from Expert Interviews

cs.CY ·2026-10-30 ·Lily Stelling, Jacob Charnock, Kevin L. Wei, David Krueger
published similarity 0.56
Abstract · Published at AIES 2026

We conducted 37 semi-structured expert interviews to examine how frontier AI evaluations reduce societal-level AI risks. Evaluations have become central to AI risk governance, despite their shortcomings being widely critiqued. Still, the case for how they reduce risks in practice remains largely implicit. Drawing on expert accounts, we reconstruct this case by identifying the mechanisms through which evaluations are expected to reduce risk, the assumptions underpinning those mechanisms, and the challenges those assumptions face. Our findings suggest that `evaluation-based governance' could contain foundational gaps which undermine risk-reduction strategies that depend on evaluations. Using thematic analysis, we identified three key pathways through which evaluations could reduce risk: by (i) making societal-level risks concrete to decision-makers and the public (raised in 18 out of 37 interviews), (ii) serving as a basis for governance commitments and regulation (18), and (iii) incentivizing a better risk culture within AI companies (16). However, these pathways often depended on shared assumptions that participants themselves doubted. The main challenges included, in order of frequency: (i) diminishing informativeness of evaluation results as model capabilities improve (21), (ii) measurement limitations that may be inherent to the evaluation paradigm (18), (iii) a lack of guarantees that evaluation results will be acted upon by companies or governments (16), and (iv) the degree of control and discretion AI companies retain over evaluation design, disclosure, and response (12). Participants also noted that evaluation-based governance could prove counterproductive, by functioning as a permission structure for developers' continued capability improvement (4). We finish by discussing the implications of our findings for evaluation-based governance, and what would improve its prospects.

0.32

Beyond accuracy: what AI assurance needs to establish in public services

cs.CY cs.CL ·2026-11-04 ·Louise Yorke
similarity 0.56
Abstract · 20 pages, 6 tables, 2 figures

Public bodies increasingly rely on AI-produced evidence, including classifications, summaries and assessments. These outputs can inform policy decisions and service delivery, making their reliability a matter of public accountability. Establishing whether this evidence can be relied upon requires testing its outputs and examining how organisations decide whether those tests are sufficient. We combine technical evaluation with analysis of AI governance requirements, informed by measurement theory and management-based regulation. We compare eight GPT-4o configurations classifying 1,200 UK Prime Minister's Questions into 21 policy topics, against human-coded labels and across unchanged repeats. Accuracy ranges from 68.2% to 70.3%, yet 24.3% of questions receive different labels, with changes in topic shares and rankings. Configurations agree more with one another than with human coding: agreement does not guarantee correctness. Under the implemented score-extraction procedure, the proportion flagged for human review ranges from 0.9% to 41.0%. We analyse 37 UK and EU policy documents. Both sets expect testing and oversight, but reporting expectations do not consistently cover changing answers, shared errors, selective evaluation, substantive conclusions or review routing. Some UK guidance explicitly addresses stability. The UK sample emphasises public bodies checking their own use; the EU places substantial testing duties on providers while leaving them important choices about measures and acceptance thresholds. Without appropriate checks, evidence could misrepresent policy issues, redirect attention or resources, and leave errors undetected. Public policy should require organisations to justify reliance through checks beyond aggregate accuracy, clear responsibility, access to expertise and scrutiny of results.

0.24