CS Paper Radar

Invisible Agents, Uninformed Patients: Towards Responsible Deployment Of Autonomous AI Diagnostic Agents In Sub-Saharan Africa

Percy Brown, Kweku Yamoah

cs.CY ·submitted 2026-08-21 17:34 UTC · v1

Abstract

Autonomous AI diagnostic agents, systems that analyse patient-specific clinical data and produce diagnostic outputs or triage decisions without mandatory real-time human review, are increasingly deployed across eHealth platforms in sub-Saharan Africa at a pace that has outrun the governance infrastructure needed to oversee them. While significant bodies of work address AI accountability, transparency and explainability in healthcare, existing frameworks are largely clinician-centered and assume regulatory conditions that do not uniformly exist in low-resource settings. A patient-centered analysis of the disparity in patient awareness regarding autonomous agents, which results in a structural accountability gap, is mostly missing from the literature. This paper synthesizes existing research on informed consent, algorithmic accountability, and explainable AI to highlight three distinct challenges introduced by deploying AI agents in the sub-Saharan African context. Drawing on three documented deployment cases, including computer-aided tuberculosis detection in Tanzania, diabetic retinopathy and TB screening in Zambia, and mobile health chat-bot triage in Ghana, it demonstrates that these gaps are already present in active deployments across the region. In response, the paper proposes three foundational principles; agent-aware informed consent, human override as a structural requirement and contextually adapted explainability. This triad of principles lays a practical minimum standard for developers, health system administrators and policymakers in contexts where formal AI regulation remains nascent.

Comment
10 pages, 4 figures

Closest papers

Nearest neighbours by meaning across the whole index.

From Knowledge to Legitimacy: A Philosophical Problem Discovery of AI Implementation Readiness in Public Health Disease Surveillance

cs.CY ·2026-09-30 ·Tithi Mitra, Souvik Pramanik, Most. Aysha Siddika Sumona, Ahmed Faizul Haque Dhrubo, Farhana Sharmin +1 more
similarity 0.66
Abstract · 15 pages, 3 figures, 3 tables

Despite being an emerging credible means of using artificial intelligence (AI) for improving disease surveillance via early detection of outbreaks, epidemics prediction and evidence-based decision-making, there continue to be challenges in the use of AI tools in many low- and middle-income countries (LMICs) due to various factors including fragmented health information system, digital inequality, governance problems and institutional incapability. Most of the research conducted so far has concentrated on the efficacy and accuracy of AI in terms of predicting outbreaks, with little focus on other conditions necessary for its deployment. This study adopts a qualitative problem-discovery research design, integrating thematic analysis with philosophical analysis to examine the structural and normative barriers surrounding AI implementation in disease surveillance. The analysis identifies four interrelated dimensions of implementation readiness: epistemic adequacy, distributive justice, ethics of governance, and institutional legitimacy. These dimensions provide a framework for understanding how limitations in knowledge integration, unequal digital infrastructure, privacy and accountability concerns, and deficits in institutional and public trust can constrain the practical adoption of AI-enabled surveillance. Rather than proposing another predictive model, this study develops a theoretical framework that conceptualizes AI implementation in disease surveillance as simultaneously a socio-technical and normative process. The framework provides a foundation for subsequent empirical investigation and offers a structured perspective for designing more context-sensitive, ethically grounded, and institutionally sustainable AI-enabled disease surveillance systems in resource-constrained healthcare settings.

0.21

ETHOS: Towards a Modular Ethics Framework for Clinical Multi-Agent Systems

cs.MA cs.AIcs.LG ·2026-08-15 ·Rakesh Sharma, Sydney Pugh, Cameron Beeche, Pankhuri Singhal, Rachel Wu +12 more
similarity 0.65
Abstract · Preprint of an article submitted for consideration in Pacific Symposium on Biocomputing ©2027 World Scientific Publishing Company. \url{http…

The rapid adoption of large language models has enabled the development of clinical multi-agent systems (MAS) capable of integrating multimodal patient data and supporting increasingly complex clinical decision-making. However, the deployment of these systems in real-world healthcare settings raises critical ethical concerns related to safety, fairness, accountability, transparency, and patient trust. While numerous organizations, including the World Health Organization, the National Academy of Medicine, and the FUTURE-AI consortium, have proposed ethical frameworks and governance principles for healthcare AI, these efforts remain largely conceptual. To address this challenge, we present ETHOS (Ethics and Trust through Hierarchical Oversight System), a modular ethics framework designed as a governance meta-agent that can be integrated with any existing multi-agent system without requiring changes to its underlying architecture. ETHOS translates stakeholder-informed ethical requirements into executable runtime oversight through a layered governance approach consisting of deterministic checks, contextual reviews, and a final ethics critic. These components continuously evaluate intermediate reasoning steps and final outputs, enabling the system to identify ethical risks, request revisions, or suppress responses that fail predefined safety and trustworthiness criteria. We demonstrate ETHOS within a hepatology clinical decision-support MAS. Results show that ETHOS improves decision reliability by detecting incomplete, inconsistent, or out-of-scope evidence and appropriately increasing abstention when safe recommendations cannot be supported. By embedding ethical governance directly into system operation, ETHOS provides a practical and auditable mechanism for transforming high-level AI ethics principles into deployable safeguards.

0.17

From AI-Enabled Remote Patient Monitoring Toward Agentic Robotics: Embodiment, Governance, and Care Integration

cs.RO cs.AIcs.MAcs.SE+1 ·2026-11-13 ·Ivana Ragusa, Lorenzo Guiducci, Filippo Cavallo
published similarity 0.64
Abstract · 34 pages, 6 figures, 3 tables. Accepted for publication in Annual Review of Control, Robotics, and Autonomous Systems, Volume 10 (2027)

AI-enabled remote patient monitoring is evolving beyond passive data collection toward agentic AI systems that interpret patient states, coordinate care, and initiate action across domestic and clinical settings. This structured scoping review finds that mature agentic care has not yet emerged: Most systems remain human led, while stronger decision-action coupling and embodied AI remain largely confined to prototypes and pilots. Healthcare robotics makes the transition from monitoring to action especially visible by translating AI-generated recommendations and decisions into speech, movement, prompting, physical assistance, escalation, and care coordination, making embodiment, clinical integration, human oversight, and governance fundamental design requirements. Further progress requires safe and interoperable care architectures in which intelligent components operate through clinically validated and accountable workflows. We propose agentic robotics as a future direction for supervised systems embedded in the care ecosystem, connecting AI agents and robots with clinicians, caregivers, and patients through auditable workflows that can be interrupted and remain subject to human oversight and explicit AI governance.

0.37

Understanding Users' Security, Privacy, and Trust Concerns in Healthcare AI Systems

cs.CR cs.CY ·2026-11-15 ·Tushar Nayan, Mohammed Al Duniawi, Song Fang, Aokun Chen, Arjun Arunasalam +1 more
similarity 0.62
Abstract

Artificial intelligence is increasingly integrated into healthcare systems, from clinical information retrieval to workflow automation and decision support, which introduces new security, privacy, and trust risks with direct implications for patient care. While prior work has examined these risks from technical and regulatory perspectives, less is known about how these risks are naturally discussed in online communities. To address this gap, we conduct a large-scale analysis of 14 medical communities on Reddit and extract 6,514 relevant posts. Through systematic qualitative coding and LLM-assisted classification, we investigate concerns expressed in healthcare AI discussions, their prevalence, and how they evolve over time. Our analysis reveals that users perceive healthcare AI as expanding traditional healthcare trust boundaries and attack surfaces through integrations with electronic health records, third-party services, cloud platforms, and automated clinical workflows. Posts express concerns about over-privileged access to sensitive records, opaque data flows, unauthorized secondary use, and unsafe automation can amplify the consequences of errors and misuse. We identify four attitudes toward these risks: exploring questions of accountability and harm, cautiously accepting AI with stronger safeguards, skepticism about system trustworthiness, and resignation to growing data exposure and automation. Longitudinal analysis shows that SP&T discussions have increased significantly, with shifts coinciding with major healthcare AI deployments, privacy incidents, and regulatory developments. These findings illuminate changing trust relationships and data-governance challenges, informing the design and governance of safer, more transparent, and accountable healthcare AI systems.

0.43

Examining the Vulnerability of Multi-Agent Medical Systems to Human Interventions for Clinical Reasoning

cs.AI ·2026-09-02 ·Benjamin C Liu, Dillon Mehta, Rishi Malhotra, Adam Zobian, Yong Ying Tan +5 more
similarity 0.61
Abstract

Human interventions at fault points can alter the diagnostic accuracy of multi-agent medical systems. We defined fault points as moments in AI agent conversations, in which an agent's reasoning became most vulnerable to external influence. Using the MedQA dataset, this study analyzed simulated doctor-patient conversations to measure how interventions shifted reasoning and accuracy. Correct intervention methods showed an improvement in baseline diagnostic accuracy of up to 40%, while incorrect or bias-related interventions degraded performance by up to 6% and increased diagnostic drift and uncertainty. Beyond performance changes, our analysis revealed behavioral similarities between cognitive biases in simulated agent environments and real-world clinical practice. Examples included premature closure and susceptibility to misleading cues. Overall, these findings demonstrate that identifying and guiding fault points with human interventions may provide a mechanism for improving diagnostic robustness in multi-agent medical systems.

0.27

Between Niti and Nyaya: A Society-Centered Framework for AI Governance

cs.AI cs.CYcs.HC ·2026-11-12 ·Snehalkumar `Neil' S. Gaikwad, Punya Aragula, Shiva Kruthi Palpanoori, Pratik Patekar
hot direction similarity 0.60
Abstract

Governments are making decisions about AI before they can measure its public effects, and those decisions determine whether a person can learn that AI influenced an outcome, challenge it, and see it corrected. Research has documented what AI governance produces: principles, strategies, registers, and audits. However, we lack shared language for understanding AI policy while it is still being made. We introduce the Society-Centered AI Governance Framework, which gives policymakers, researchers, and the public a way to assess instruments in any jurisdiction by whether a person can get notice, challenge a decision, and see it corrected. It reads a jurisdiction's instruments together by date and binding status, separates an agency duty from a right a person can invoke, and traces which later instrument must enforce each commitment. We evaluated it in North Carolina using interviews with nine AI Leadership Council members, conducted before and after the state's 2026 AI roadmap, and a provision-level reading, with human-reviewed model-assisted coding, of 49 AI governance documents from 19 states. For the state's own AI systems, we find visibility-first governance: visibility duties are in effect, notice exists as an agency duty with no right a person can invoke, and AI-specific challenge and correction remain advisory. The roadmap assigns owners and dates to the missing protections, so the gaps are documented but not yet closed. Recent federal preemption proposals focus on AI development, so recourse for a state's own systems is likely to remain a state decision. We identify "legitimacy before protection" as a testable risk of visibility-first governance and name the milestones that would test it. More broadly, judging AI policy by who can act on what it makes visible lets the field assess recourse while policy is still forming, before harm occurs.

0.19

AI Deployment Accountability Engineering: A Vision for Accountable AI in Safety-Critical Socio-Technical Systems

cs.AI cs.CRcs.CY ·2026-09-13 ·Murat Kantarcioglu
published similarity 0.59
Abstract · Appeared in ACM AI Summit 26, Atlanta, GA as a vision paper

Artificial intelligence systems are rapidly becoming critical components in healthcare, finance, public services, and other safety-critical domains. Yet the engineering practices used to evaluate these systems remain predominantly model-centric, emphasizing properties such as accuracy, robustness, fairness, and interpretability before deployment. These properties are necessary but insufficient once an AI system operates within an ever changing socio-technical environment characterized by distribution shifts, institutional constraints, human feedback loops, privacy requirements, and interactions among multiple AI agents. This vision paper introduces AI Deployment Accountability Engineering (ADAE), a proposed AI engineering subdiscipline concerned with establishing measurable, continuous, and actionable accountability for deployed AI systems. ADAE treats accountability as a deployment-layer property rather than solely as a property of an individual model. It seeks to determine whether an AI-enabled system continues to operate within acceptable risk limits, identify the contexts in which failures emerge, attribute failures across interacting technical and human components, translate technical failures into downstream consequences, and support timely intervention. We articulate a research agenda built around four interconnected pillars: structured discovery of context-dependent failure modes, privacy-preserving accountability measurement, system-level risk analysis for agentic AI, and translation of technical failures into operational, and institutional risks. The broader goal is to establish foundational principles, mathematical tools, and system architectures for accountable AI deployment across safety-critical applications.

0.35

VITAL: A Framework for Validating Agentic AI in Pharmaceutical Drug Development and Clinical Trial Operations

cs.AI ·2026-11-17 ·Sandeep Burugupalli, Pedro Matos-Filipe, Andrea Hita Ardiaca, Adam Taylor, Alex Grigorenko +8 more
similarity 0.59
Abstract

Current assurance approaches do not establish whether evidence is sufficient to validate, monitor and govern agentic artificial intelligence (AI) within a pharmaceutical context of use. Plausible outputs may conceal failures in retrieval, reasoning, orchestration or human-control points, where reference standards are unavailable. We developed and operationalised VITAL (Validation by Intended use, Traceability, Autonomy and Lifecycle evidence), a validation framework for agents in clinical development, through landscape analysis, regulatory and standards mapping, expert interviews and multi-stakeholder refinement. VITAL integrates context of use, controls, behavioural validation and lifecycle evidence within a risk-adjusted framework. We applied VITAL across epidemiology and treatment-landscape analysis, site selection, enrolment rescue and trial-design trade-off support to translate autonomy, reversibility, transparency and patient impact into proportionate validation activities, evidence requirements and auditable reconstruction of system behaviour. VITAL may inform evidence requirements and lifecycle accountability for higher-risk pharmaceutical uses; evaluation should assess reproducibility across assessors, organisations and system configurations.

0.25