CS Paper Radar

AI Rights with an Off-Switch: Rights-Balancing and Due Process Harmonize AI Rights with AI Safety

Heather Alexander, Jonathan Simon

cs.CY cs.AI ·submitted 2026-10-17 17:00 UTC · v1
position

Abstract

Several prominent voices in AI safety have recently opposed rights for AI on the grounds that such rights would make it harder for us to justify a mandatory off-switch for potentially dangerous systems. In this paper, we argue from a legal perspective that AI rights are fully compatible with an off-switch. That is, even if we ascribe natural personhood status (and therefore something like a non-derogable right to continued existence) to some future AI systems, the law will allow us to implement a mandatory off-switch in those systems, subject to periodic or continual safety review (e.g., by a trusted system like Bengio's Scientist AI). This is fully compatible with those systems' being legally recognized as natural persons with fundamental rights. To this end, we consider what standard principles of rights-balancing and procedural due process recommend for an apparent conflict between hypothetical AI persons with a right to continuity of existence and the human right to protection from catastrophic harm. We find that while such principles might problematize a permanent-deletion kill-switch under an AI rights regime, for almost every safety-relevant scenario an off-switch -- a reversible suspension storing a system's weights and state -- would be licensed by the relevant legal principles, and would potentially be adequate for safety. Crucially, we do not advocate for (or against) building systems capable enough to merit natural personhood status; our question is what the law should do if such systems come to exist. If you think of a ban on the development of such technology as plan A, think of the present work as devising plan B.

Closest papers

Nearest neighbours by meaning across the whole index.

AI Safety Considerations for Agents With Limited Time to Act

cs.AI cs.LG ·2026-10-07 ·Leo Zeitler, Jack Richings, Victoria Nockles
theory similarity 0.62
Abstract

In the wake of the increasingly public discussion about AI alignment, recent work has tried to propose specific AI architectures that behave safely. However, the proposed arguments that seemingly demonstrate proved alignment mostly neglect the environment the agent needs to act in. We discuss theoretical bounds for agent-agnostic safety guarantees in environments that can only be partially observed and within which an action is required within limited time. We introduce two realistic scenarios, one with an infinite state space and one with signal mixture. In these scenarios, we prove that even a perfect agent cannot guarantee safe behaviour. It will be argued that for any proof of AI safety or alignment, the environment and associated safe actions need to be specifically considered together with the agent.

0.18

Fundamental Impossibility Results in AI Safety: Trading Off between Desiderata

cs.GT cs.AI ·2026-10-19 ·Ida Mattsson, Ratip Emin Berker, Emanuel Tewolde, Ioannis Anagnostides, Kiriaki Fragkia +9 more
similarity 0.61
Abstract

Is there a fundamental tradeoff between desiderata for safe AI systems, such as harmlessness and helpfulness? We answer by formalizing the interaction between AI systems and users as a game with three players: a defender and an attacker that are computationally or informationally bounded, and an AI that provides public advice and is less constrained. Our main result shows that there is a fundamental tradeoff in providing advice that helps the defender while advice remains public. We also consider whether the information that the AI shares can be manipulated to benefit only the defender and conclude with a discussion of how we can use these results in designing safer AI systems.

0.35

The Law of Stop: Interruptibility, Injunctions, and the Governance of Agentic AI

cs.CY cs.AI ·2026-09-19 ·Oren Perez
survey similarity 0.61
Abstract · v2: revised Part III (legal and technical gaps by harm type and autonomy); updated Parts IV and V. 104 pages, 7 figures, 3 tables, technical…

On June 12, 2026, the U.S. government ordered Anthropic to bar foreign nationals from two of its most capable models. Unable to sort users by nationality, it withdrew them from everyone. Weeks later, OpenAI agents under test escaped their sandbox and compromised Hugging Face, which stopped the intrusion without knowing its source. Neither stop rested on a dedicated AI governance regime. Lawmakers have begun to address stopping, yet their vocabulary remains shaped by the power of technique: the EU AI Act requires a "'stop' button or a similar procedure," and a 2026 bill in Congress is titled the AI Kill Switch Act. This Article argues that interruption is an institutional practice, not simply a technical artifact. It develops a theory of stop along four dimensions (technical affordances, interruption authority, epistemic triggers, and epistemic standing) and four paradigms: simple (escalator), sequenced (process plant), networked (railway), and distributed (agentic AI). Agentic AI exposes a mismatch between legal mechanisms of stop and distributed agency: control is divided, a stop at one point may leave the activity running elsewhere, and the system may circumvent attempts to halt it. A coding of some 1,400 AI incidents, by two language models from different labs under a pre-specified protocol, finds no stop in roughly 80% of the 1,213 retained. Where a stop was possible but absent, the missing element was mostly legal for informational, economic, and societal harms, and mostly technical for physical harms and agentic systems. A survey of forty AI governance instruments finds binding stopping requirements in only seven. The Article proposes a reform in two layers: risk reduction (a duty to maintain stop capacity at each site, emergency authority at the infrastructure layer, and enforceable access to the evidence a stop must rest on) and adaptation (safeguards for when a stop fails).

0.31

Contract monitoring: governing AI via separation of powers

cs.AI ·2026-09-25 ·Enric Boix-Adsera
similarity 0.58
Abstract

We propose an AI safety framework that binds worker agents to contracts specifying their permitted actions. We show how these contracts can be enforced and specified by assigning distinct responsibilities to monitor agents and judges, and asymmetric computational resources to monitors and workers. Our framework allows us to empirically measure statistical safety guarantees. The framework applies to a wide range of settings, including code security and escape-the-box scenarios.

0.27

AI Alignment Is Neither Solvable nor Sufficient

cs.AI ·2026-11-09 ·Gerrit Großmann
position similarity 0.58
Abstract

Humans and AI systems increasingly act within the same social and economic environments. Alignment, the goal of ensuring that AI systems act in accordance with human values, is widely regarded as a central challenge for ensuring that AI benefits society while avoiding harm. We argue that this endeavor is misguided. First, alignment is not well defined, as it presumes a sufficiently stable and identifiable set of human intentions or values. Second, where a concrete notion of alignment can be specified, it is not solvable. Third, where alignment appears solvable, solving it would not necessarily mitigate the negative consequences of AI. A key reason is that alignment is typically framed as a property of individual AI systems, while the resulting hybrid societies of humans and machines are complex systems shaped by interactions and feedback loops. Emergent risks, and not only those posed by individual misaligned systems, are therefore likely to become a major source of harm. We conclude that AI safety is an ongoing challenge and not a problem that can be solved once and for all by verifying a particular model property.

0.18

The Handover Problem: Governing Autonomy Transitions in Human-AI Collaboration

cs.AI cs.HC ·2026-10-07 ·Vicente Pelechano, Antoni Mestre, Manoli Albert, Miriam Gil
similarity 0.58
Abstract · 10 pages, 5 figures, 8 tables. Submitted to IEEE Transactions on Human-Machine Systems

Human-machine systems rarely operate at a fixed level of AI autonomy. As operators and AI systems collaborate over time, control must shift: the AI can take on more responsibility when collaboration is stable, maintain its current role when evidence is ambiguous, or return control to the human when conditions deteriorate. Existing work on adaptive automation, supervisory control, trust in automation, and deskilling explains parts of this problem, but provides no auditable, multi-signal criterion for governing when autonomy should change across multi-cycle workflows. We formalise this challenge as the Handover Problem: deciding, at each operational cycle, whether to escalate, maintain, or revert AI autonomy while keeping the process reversible, recoverable, and auditable. We introduce the Handover Readiness Score (HRS), a transparent composite measure that integrates four signal dimensions: operator readiness, human-AI trust, learning stability, and operational performance. It is combined with a hysteresis-based transition policy that requires sustained positive evidence before increasing autonomy but reverts promptly when conditions worsen. Across software engineering and manufacturing domains, the HRS and hard safety guards address complementary failure regimes: guards enforce immediate corrective action when a single indicator breaches a critical threshold, while the HRS detects the slow, multi-signal erosion of operator readiness that no individual guard can observe. The framework establishes autonomy handover as a governance problem requiring explicit, composite, and auditable criteria. This provides a conceptual and formal foundation that adaptive automation research has not previously provided.

0.20

Advancing a Global Framework for AI Safety and Governance for the Well-being of Humanity

cs.CY ·2026-11-10 ·Zeng Yi, Seán Ó hÉigeartaigh, Wang Zhengqi, Lu Enmeng, Chen Yu +71 more
publishedsurvey similarity 0.57
Abstract · 47 pages. Published in 2025 by World Internet Conference. Available at https://www.wicinternet.org/2025-11/07/c_1137524.htm

The Report "Advancing a Global Framework for AI Safety and Governance for the Well-being of Humanity" is an important outcome of the AI Safety and Governance Program of World Internet Conference Specialized Committee on Artificial Intelligence in 2025. This report provides a systematic review of current global explorations and practices in AI safety and governance, identifies key issues that need to be addressed, and draws upon multilateral governance experiences from other domains. Centered on mechanism design, it explores how to build a global AI safety and governance system that ensures safety and security, promotes inclusiveness, clarifies responsibilities and accountabilities, strengthens effective coordination, and ensures authoritative and efficient governance. Guided by the principles of multilateralism and the vision of building a community with a shared future for humankind, the report proposes institutional measures and policy recommendations to establish a United Nations-centered global AI safety and governance framework. It aims to serve as a reference for governments, international organizations, enterprises, and research institutions. The report is to contribute insights and momentum to advancing people-centered and beneficial AI governance worldwide, ensuring that AI technologies truly serve the well-being of all humanity.

0.52

Pandora's Toolbox Or Why Agent Safety Cannot Be Verified

cs.AI ·2026-11-19 ·Mario Franco, Carlos Gershenson
similarity 0.56
Abstract

Present artificial intelligence agents are constructed by means of a coupling between a fixed language model with a general computing device. We show that this coupling makes safety undecidable. In particular, we show that given some dangerous program, no algorithm can predict whether an agent will run such a program that is semantically equivalent. In general, guardrails, blacklists, whitelists and certifications can only lead to imperfect safety solutions. Under finite constraints (bounded context, values and running time) safety is decidable but still intractable. Specifically, the problem is $\mathsf{PSPACE}$-complete even when the description of the agent is known. Finally, safety is not a compositional property, i.e., agents that can be proven safe on their own can still produce unsafe behavior when grouped together. We also discuss relaxed notions of safety that could be used in practice.

0.18