CS Paper Radar

From Policing Provenance to Designing for Evidence: Testing Four Propositions on Generative AI Against Research and Institutional Policy

Samar Ansari

cs.CY ·submitted 2026-10-31 22:54 UTC · v1

Abstract

Generative AI has made it hard to establish whether a student produced a piece of assessed work. Building on the argument that validity matters more than cheating, this paper tests four propositions drawn from a strongly stated position: that redesign is more sustainable than detecting or banning AI; that asking students whether and how much AI they used yields low-value information; that unaided ability should be tested directly in controlled conditions; and that the same logic applies to peer review. The propositions are tested against two purposively sampled, coded corpora: 41 research and sector sources and 55 university and sector policy documents. Coding was by a single coder and the policy sample is UK-weighted, so counts describe the sample, not the sector. The propositions hold unevenly. Redesign over detection is well supported, but only as redesign plus assurance: 4 of 55 documents prohibit AI by default, though about 10 of the 55 are publisher or editorial policies that govern authors rather than student assessment; of the 23 that take a position on detection, 16 reject detectors and one requires them. Testing unaided ability is supported as one lane within a programme, not as a return to invigilation. The disclosure proposition is complicated: institutions treat disclosure as valuable, yet the limited evidence suggests many students do not complete required declarations. The peer-review extension is plausible but under-examined. The field is converging on graded permission scales and two-lane designs, and 31 of 55 documents define named categories of use. From these findings the paper builds a nine-step decision chain that extends outcome-first design logic: the role of AI is decided after the evidence an outcome requires. It also separates design moves that add temporary friction from those that provide assurance. A worked example and the full policy matrix are included.

Closest papers

Nearest neighbours by meaning across the whole index.

Dynamic Evidence Collection Ecosystem for Assessment Integrity and Authentic Competence

cs.HC cs.AI ·2026-08-17 ·Rajan Kadel, Bellal Hossain, Samar Shailendra, Bushra Naeem
hot direction similarity 0.69
Abstract · ISET 2026

Generative Artificial Intelligence (GenAI) can produce high-quality essays, code, and design artefacts, challenging the validity of conventional assessments that rely on single-point submissions and product-only grading. This paper proposes a design framework called "Dynamic Evidence Collection Ecosystem" that shifts assessment toward continuous, authentic, multi-source evidence of student learning over time. The framework collects process evidence through iterative artefacts, design logs, activity rounds, self-reflection, and peer collaboration, supported by an AI-enabled layer for learning analytics, formative feedback, and transparency. The approach is grounded in recent assessment-redesign scholarship in AI-rich contexts and aligned with contemporary views of authenticity in assessment. This paper builds on the hypothesis that academic integrity is strengthened when it is treated as an assessment design rather than as an AI detection problem. The tools have limitations and risks of use that carry academic penalties. This paper presents an implementation scenario to support institutional adoption.

0.33

Evaluation in the Age of AI: Output as Evidence of Learning

cs.CY ·2026-08-23 ·Md Zarzees Uddin Shah Chowdhury, Samin Rahman Khan
position similarity 0.66
Abstract · 12 pages, 1 figure, 1 table

The rapid adoption of artificial intelligence (AI), particularly large language models (LLMs), has fundamentally disrupted how learning is demonstrated and evaluated in higher education. Tasks that once served as proxies for understanding-such as writing essays, solving problem sets, or producing computer code-can now be generated superficially by AI systems with minimal human effort. This paradigm shift raises a critical ethical question: how should learning be evaluated when traditional indicators of competence are easily outsourced? This paper examines the ethical challenges of educational evaluation in the age of AI from a university-level perspective. We argue that the core problem extends beyond academic dishonesty to a deeper misalignment between assessment practices and the learning outcomes they are intended to measure. Evaluation regimes that rely on artificial constraints risk measuring compliance, access, or concealment rather than genuine understanding, reasoning, or judgment. By analyzing institutional responses and presenting empirical survey data, we highlight the need for alternative assessment models that emphasize process over product. The goal is to establish ethically informed assessment strategies that preserve student agency and accountability in an automated age.

0.23

Wording Matters: Understanding How GenAI Policy Rules Shape Teachers' and Students' Workflows and Perceptions

cs.HC ·2026-11-08 ·Sabiha Mehreen, Xiangzhe Yuan, Rishab Nithyanand, Naimul Hoque
similarity 0.63
Abstract

The growing use of generative AI (GenAI) in higher education has led institutions to develop policies governing its applications. However, we have little knowledge about how teachers and students adapt and adhere to the policies. In this paper, we analyzed GenAI policies from 101 U.S. universities, identifying nine recurring rules for GenAI use, and interviewed 20 students and 20 faculty members about the rules. We find that the wording of the rules can change how students perceive and act on the rules while introducing different challenges that teachers face in enforcing them in practice. For example, rules such as "Passing off AI-generated content as your own constitutes academic dishonesty" and "Students are accountable for any inaccurate, biased, or unethical content" both outline the acceptable use of AI, but the former can make students hide their use of AI out of fear of unwanted scrutiny, whereas the latter can encourage students to take responsibility and ownership of their use of AI and be more transparent. Students described uncertainty about acceptable use, fear of penalties and unfair treatment, and stigma that discouraged acknowledging AI use. Teachers took on approval workload, had to interpret acceptable use, and lacked reliable ways to verify compliance. We summarize these impacts in a policy-impact matrix linking each rule to its effects on teacher and student workflows and perception.

0.28

Mapping the Authorized Boundary: A Comparative Policy-Vignette Study of Generative AI Governance in Australian Higher Education

cs.CY ·2026-08-30 ·Biranchi Poudyal
similarity 0.63
Abstract

Australian universities regulate students' use of generative artificial intelligence (GenAI) through overlapping policies, procedures, guidance, and assessment instructions, but these environments may classify identical conduct differently. We applied 15 standardized student-use vignettes to the public policy environments of 20 Australian universities and produced 300 university-case classifications. We analyzed binding instruments (Layer A) separately from the full official environment (Layer B), then classified each combination as clearly permitted, permitted with conditions, potential policy breach, clearly prohibited, or indeterminate. We measured cross-university divergence with normalized Shannon entropy. We classified 120 combinations (40.0%) as clearly prohibited, 97 (32.3%) as potential policy breaches, 27 (9.0%) as permitted with conditions, and 56 (18.7%) as indeterminate; none met the strict threshold for clearly permitted. Disclosed language rewriting and a disclosed AI-drafted paragraph produced the highest divergence, whereas an explicit assessment prohibition produced unanimity. Binding instruments remained silent on GenAI in 100 combinations, and guidance clarified 88. The primary researcher led the coding with AI assistance and manually reviewed all 201 queued rows. An independent human second coder, who used no AI assistance, coded a blind 75-row sample. Overall agreement reached 57.3% (unweighted Cohen's kappa = 0.395; bootstrap 95% CI [0.244, 0.539]). The findings distinguish permission-gated from disclosure-based architectures and show that written policies regulate the retention of AI-generated text more clearly than process-only assistance does. Comparative policy-vignette testing evaluates whether written governance supports defensible classifications; it does not predict misconduct or enforcement decisions.

0.18

Generative AI and the Crisis of Higher Education Assessments: Examining 'AI-Proofing' Strategies for College-Level Assessments

cs.AI ·2026-11-16 ·Bonam Mingole, Davis Yadav, Hui Min Lee, S. Shyam Sundar, Amulya Yadav
similarity 0.62
Abstract

Higher education is confronting an academic-integrity challenge unlike any in its history. Students can now outsource homework, take-home assignments, and online examinations to large language models (LLMs) that complete them in seconds. Existing research on evaluating LLM performance on academic assessments has focused largely on standardized examinations, or on questions from college-level courses drawn from a single discipline. Far less is known about how LLMs perform on college-level assessment questions spanning multiple disciplines, that instructors have purposely designed to be AI-resistant, and about which features of such questions, if any, meaningfully constrain LLM performance. We address this gap by collecting 75 questions from authentic college coursework assessments, spanning 17 disciplines, which were contributed by 37 faculty members from ten U.S. universities and one U.K. university. Of these, 61 questions were intentionally designed by faculty members to be 'LLM-proof', while the remaining 14 questions were sourced from online courses. We then (i) evaluate three state-of-the-art LLMs' ability to correctly answer these questions with and without supplementary course materials; (ii) identify five strategies, through thematic analysis, that faculty used to make their questions harder for LLMs to solve; and (iii) test whether these strategies impact LLMs' ability to answer the questions accurately. Our experimental results show that the best-performing model scored approximately 83% without supplementary course materials and 85% with them, and we observed noteworthy differences across models, answer formats, and instructor tweaking strategies. We discuss implications for assessment design, academic-integrity policy, and the broader ethics of deploying LLMs in educational settings.

0.21

Think Before You Accept: Can Written Justification Reduce Uncritical Uptake of AI Writing Suggestions?

cs.HC ·2026-09-20 ·Yan Tao, Jennifer Meyer, Rene F. Kizilcec
similarity 0.62
Abstract

Generative AI can offer students useful feedback, but its value depends on judging which suggestions are accurate and relevant. Prior research shows that strategic friction during human-AI interactions can promote critical uptake, but how to effectively implement such friction in academic contexts remains unclear. We examine whether requiring students to justify decisions to accept or reject AI suggestions can mitigate uncritical uptake in academic writing. In a randomized experiment embedded in a course activity (N=129), students wrote a data analysis proposal, received mixed-quality AI revision suggestions, and decided whether to accept or reject them. Students required to provide written justifications were 24 percentage points less likely to adopt flawed suggestions (65% vs. 41%), with no reduction in acceptance of sound suggestions (81% vs. 86%). However, thematic analysis revealed superficial engagement in the justification task and gaps in metacognitive monitoring and domain knowledge.

0.26

SoK: Grading Open-Text Legal Answers, from Law School Exams to Trustworthy Rubric Judges

cs.CL ·2026-10-14 ·Sebastian Nagl
survey similarity 0.60
Abstract · preprint

Law students are examined through open-text answers: case solutions, essays and memos that apply the law to a set of facts. Such answers are expensive to grade, human graders disagree widely on them, and large language models (LLMs) now offer to grade them at scale, both in teaching and in the benchmarks that test legal AI. This paper systematizes how open-text legal answers can be evaluated. We derive eight desiderata from educational measurement and from the law that governs exams, among them reasons, credit for defensible alternative solutions, robustness to gaming and human oversight. We then compare eight evaluation paradigms against them, from unstructured human grading to LLM judges with instance-specific rubrics and pairwise comparison. Three findings stand out. A rubric written for the task is the strongest lever for agreement between an LLM judge and human graders, but studies measure that agreement against different references and rarely report how far the human graders agree with each other. The central validation gap lies where examination law is most demanding: defensible alternative solutions, follow-on errors, and the examiner's correction of model judgments. Rubric-centred grading is the most auditable family, because it makes the standard, its application and the final decision checkable. From this we derive the trustworthy rubric judge: an instance rubric that credits arguments and alternative paths, evidence checked in code, probes, repeated runs, calibration against independent human grades, logging, and an examiner who reads every script. Today the evidence supports such a judge for formative feedback and for benchmarks, and as an assistant to the examiner, not a replacement, where credit is at stake.

0.17

Beyond fabrication: Misattributed numbers in papers written by AI research agents

cs.AI cs.CLcs.DL ·2026-10-28 ·Mosab Alfaqeeh, Omar Mohammed Horani, Mou'th Hourani, Hashem Alfaqeeh
benchmarkcode/data similarity 0.59
Abstract · v2: corrected author names; clarified the second-judge comparison and two table captions. 43 pages, 5 figures, 13 tables. Code and data: htt…

Verifying numerical claims in research written by AI agents is an important but challenging problem. Recent benchmarks show that many agent-written papers fail integrity checks, but such verdicts, often produced by a single LLM judge, identify failure without explaining its nature, and existing checks either test a paper for internal consistency or test whether a reported value appears somewhere in an agent's output. We re-analyze all 540 released judgments from a recent public benchmark of AI research agents (nine agents, fifteen open research problems) and manually label 256 numerical claims from 18 submissions. Failures concentrate on alignment between the paper and the underlying work rather than overt cheating, and about four in five unsupported claims correspond to values in the agent's own files that come from the wrong experiment, metric, or run. We introduce a deterministic claim-to-evidence checker that scores whether the surrounding context identifies the claimed result and calibrates each claim against numerical decoys. With its rules fixed before labeling, it certifies claims with over 90% precision against blinded annotators (78 of 85; Cohen's kappa = 0.89), while verifying only a subset of claims; most claims it cannot verify are nevertheless traceable, so failure to verify is not evidence of falsehood. An LLM verifier given the same evidence recovers more traceable claims but accepts about three times as many misattributed ones, so the two are best used together. Applied to 99 submissions, the checker reveals large differences in traceability across agents, and on 30 submissions a second judge configuration, with a different model and harness, orders the agents' vendor groups as the official judge does while passing twice as many papers. These findings motivate provenance-first auditing of AI-produced research.

0.37