CS Paper Radar

A Sociotechnical Review of Algorithms in Health Systems: Technical, Cost, and Human-Centered Considerations

Victoria Chui, Kelly McConvey, Shion Guha

cs.HC ·submitted 2026-09-18 17:56 UTC · v1

Abstract

Artificial intelligence (AI) applications in healthcare are becoming increasingly prevalent, to assist health systems, providers, and patients with tasks such as decision-making, risk prediction, and diagnosis. This increasing computational potential brings AI applications to the forefront of workplace decision making, often without full consideration of subsequent computational, organizational, and social costs. These applications are leveraged to reduce healthcare costs and increase efficiency of daily tasks, with model-related costs being considered at varying levels of granularity. To understand these trends, we critically analyze 114 papers to examine how cost-aware AI models have been developed for health systems. We explore the data, method, and outcome choices of these models, as well as their intersection with cost and human-centered concerns, highlighting the gaps in rigorous sociotechnical model design. From these trends, we define model costs and subsequent dimensions, presenting insight into those studies reporting financial, computational, organizational and/or social measures. Further, we critique the benefits and challenges of evaluating model-related costs and sustainability concerns when developing AI models for health systems.

Closest papers

Nearest neighbours by meaning across the whole index.

FLARE: A Systematic, Uncertainty-Aware Framework for Evidence-Based Adoption of Artificial Intelligence in Healthcare

cs.AI ·2026-08-24 ·Jacob Idoko, Siddhartha Paudel, Mariana Bento, Roberto Souza, Gouri Ginde
similarity 0.63
Abstract

Artificial intelligence is increasingly being introduced into healthcare workflows, yet most evaluations emphasize model accuracy rather than whether adoption is economically worthwhile in real clinical settings. This study proposes FLARE, a systematic and uncertainty-aware framework for evaluating the financial and operational implications of adopting AI in healthcare. FLARE combines fuzzy logic, time-driven activity-based costing, and return on investment analysis to estimate the cost of clinical service delivery, the cost of AI development and operation, and the economic consequences of workflow integration under uncertainty. The framework was demonstrated through an early health technology assessment case study of AI-assisted large vessel occlusion detection in the CT stroke pathway for acute ischemic stroke. The case study shows how FLARE can quantify conventional pathway cost, AI-related development and recurring costs, and AI-enabled service savings within a unified activity-based model. Under expected assumptions, the analysis identified a break-even threshold of approximately 3,992 patients per year, with positive first-year return on investment at typical annual stroke volumes of about 5,000 patients. The results further show that economic benefit depends not only on algorithmic performance, but also on patient volume, verification time, infrastructure choices, and workflow design. FLARE provides a transparent and practical decision-support framework for early-stage evaluation of AI adoption in healthcare. By making uncertainty, resource use, and implementation trade-offs explicit, it helps clinicians, administrators, and policymakers determine when AI deployment is economically viable and where operational changes may improve value.

0.33

Towards Multi-Method Evaluation of AI in Health: Integrating Benchmarks, Autoraters, and Expert Evaluation

cs.CY ·2026-11-03 ·Sarah Montgomery-Taylor, Hamsa Subramaniam, Rob Brisk, Liam Foster, Jack Ching +28 more
similarity 0.63
Abstract · 16 pages, 1 figure, 2 tables

General-purpose generative AI (GenAI) models are being rapidly adopted in healthcare, but they are challenging to assess with traditional evaluation frameworks. These conceptual, methodological, and practical challenges parallel decades of work in evaluating clinician and health system quality and safety. We describe the adaptation of methods from several fields, combined with emerging technical approaches, into a structured, multi-method evaluation framework designed to give well-rounded assessments of GenAI in health at scale. The approach first anchors evaluation to Critical User Journeys (CUJs) to help prioritize out of a large choice space, and maps these to a phased AI system lifecycle (base model development, product development, and post-deployment monitoring), since evaluation at each stage requires different methodological approaches. In this context, the approach then integrates three distinct evaluation methods (closed-question benchmarks, LLM-as-a-judge autoraters, and expert human review) across three core domains: accuracy, safety, and usefulness. Taken together, this approach provides a strategic framework for bridging the gap between benchmarking results and performance in the real world with the goal of supporting decision-making and accountability across development, procurement, and deployment for product teams, health systems, and regulators.

0.14

AI Model Design and Physician Liability

cs.AI econ.TH ·2026-08-12 ·Shujie Luan, Shubhranshu Singh, Tinglong Dai
similarity 0.62
Abstract · Forthcoming in Management Science. 66 pages, 8 figures, 5 tables; includes online appendix (pp. oa1-oa15)

Clinical algorithms can perform differently across patient groups, a concern that grows as these algorithms increasingly take the form of artificial intelligence (AI) models. In the United States, a 2024 federal rule bars healthcare providers that receive federal funds from discriminating through their use of clinical algorithms, which exposes them to liability when such use disadvantages a protected patient group. We examine how such liability considerations reshape (i) an AI firm's model design decisions that drive group-specific accuracy and (ii) a physician's decisions to use AI in healthcare delivery. The firm designs an AI model for two patient groups, and improving accuracy for the disadvantaged group is more costly. The physician (who remains the accountable decision maker) then decides whether to consult AI, weighing the reduction in clinical uncertainty against expected liability exposure when AI errors disproportionately affect the disadvantaged group. We find that this liability can induce disparate use of AI: the physician may reduce AI use overall and, over an intermediate range of liability, rely on AI less for disadvantaged patients. The effect is non-monotone. As liability increases, the physician's use of AI for disadvantaged patients first declines, then rises as the firm reallocates investment toward reducing disparity or switches to an equal-accuracy design. Mandating equal algorithmic accuracy across patient groups can then inadvertently harm both groups, because a uniform accuracy requirement distorts the firm's investment incentives and the physician's equilibrium AI-use decisions.

0.10

From Prediction to Action: Population Health World Model for Digital Medicine

cs.AI ·2026-10-17 ·Xike Xie, Yu Li, Weizhong Yang, Amalie Dyda, Shaohua Kevin Zhou
position similarity 0.61
Abstract

Digital medicine now captures data that traces individual health trajectories and reveals population health dynamics through electronic health records, claims, remote monitoring, wearable devices, mobile devices, social sensing data, medical images, etc. But much of its modeling still centers on task-specific, outcome-specific, and prediction-specific estimation. The next frontier is not simply better prediction, but action in an evolving health world shaped by multiple decision-makers, heterogeneous populations, resource constraints, and feedback in care delivery. This is why digital medicine now needs world models: not only to estimate what will happen, but to represent how actions change what can happen next -- whom to prioritize, how to allocate scarce resources, how to adapt care across subgroups, and how interventions reshape future health trajectories and population dynamics. We call such a model population health world model (PHWM): a more foundational representation of an evolving health world in which health states, individual trajectories, population health dynamics, interventions, constraints, and feedback are intrinsically coupled. Building a PHWM endows three unmet structural gaps in current digital medicine: dynamic health-state modeling, population-level intervention intelligence, and validation, equity, and governance for action-oriented use. We argue that addressing these gaps requires three minimal capabilities: representation learning under action-dependent observation, multi-scale intervention intelligence, and validation with explicit decision boundaries, while highlighting the importance of social, contextual, and public-health sensing data for action-oriented digital medicine.

0.21

"It Feels Weird to Just Rely on AI": How Experienced Healthcare Professionals Work With Decision Support Systems in Clinical Practice

cs.HC ·2026-10-29 ·Simon W. S. Fischer, Hanna Schraffenberger, Miranda L. van Hooff, Casper Reijnen, Serge Thill +1 more
similarity 0.60
Abstract

Decision support systems (DSS) assist with clinical decision-making. Little is known, however, about the decision-making process with DSS in clinical practice and the extent to which clinicians consider or rely on the predictions. Studies on human-AI decision-making usually focus on laypeople, simplified decision tasks, or one-shot interactions with prototypes that are not integrated into existing workflows. To gain insights from real-world practice, we conducted an online survey (n=12) and semi-structured interviews (n=10) with clinicians from two different medical cohorts each working with a DSS. Findings show, amongst other things, that the interaction with DSS changes as experience increases, the predictions become more important in complex cases, that decisions are discussed with colleagues, and that workflow integration affects decision-making. Our findings on human, technical, contextual, and organisational factors contribute to a socio-technical perspective on DSS reliance behaviour. We conclude by discussing the implications of our findings for the design of DSS.

0.21

Rethinking How We Evaluate Methodological Progress in Health AI

cs.LG cs.AI ·2026-09-16 ·Florent Pollet, Matthew McDermott
similarity 0.60
Abstract

Methodological progress in artificial intelligence (AI) for electronic health records (EHRs) depends on our ability to determine which algorithms work better, and under which conditions. However, such progress is thought to be hindered by difficulties in reproducibility and in defining clinically meaningful evaluation tasks. We empirically study these barriers by re-implementing 12 historical and recent algorithms within a shared evaluation framework and evaluating them on two clinical datasets, MIMIC-IV and NWICU. We compare two complementary task families: expert-authored clinically meaningful tasks and generated tasks defined from randomly sampled event codes and prediction horizons. We ask whether relative algorithms comparisons transfer across task families and datasets, whether residual task heterogeneity contains useful methodological structure, and what a controlled comparison reveals about progress over the last decade. We find that aggregate pairwise comparisons transfer strongly across evaluation settings, including from randomly generated tasks to clinically meaningful tasks and across datasets. At the same time, clinically meaningful tasks exhibit greater task-method interaction, providing preliminary evidence that task properties can help explain when particular modeling choices are advantageous. Finally, newer algorithms do not consistently outperform earlier approaches: gradient-boosted trees remain highly competitive when paired with a modern, wide and sparse representation of the EHR. Together, these results suggest that useful methodological knowledge may require less task engineering than commonly assumed, while highlighting the importance of understanding the structured heterogeneity that remains across tasks and methods.

0.42

Do AI Agents need Preventive Care? An Epidemiologic Cohort Design for Agentic AI

cs.AI cs.MA ·2026-11-09 ·Shu Jiang, Graham A. Colditz
similarity 0.60
Abstract

As AI agents increasingly perform tasks autonomously across sectors, including healthcare and biomedical research, understanding how their behaviors develop and influence outcomes over time becomes increasingly important. Existing evaluations largely focus on model capabilities and performance on predefined tasks. Here, we propose an epidemiologic cohort design for studying agentic AI, drawing on principles established in human population studies. The framework treats individual agents as members of a cohort, their underlying model as genetic makeup, accumulated work as their time scale, and naturally occurring behaviors as exposures that may influence subsequent outcomes. By following agents longitudinally, this approach enables investigation of behavioral determinants of adverse events, estimation of their potential preventable burden under causal assumptions, and characterization of behavioral changes over an agent's work life. We illustrate the design through two prospective observational cohorts examining curiosity and probing as a safety question, and data checking and analytical accuracy as a reliability question. Together, the framework and illustrative studies provide a generalizable blueprint for designing epidemiologic cohorts of AI agents.

0.26

"It's still 100% on me": Designing AI Tools for the Intersection of Physicians' Care and Legal Infrastructures

cs.HC ·2026-11-11 ·Gennie Mansi, Julia Kim, Lynn Dombrowski, Mark Riedl, Naveena Karusala
similarity 0.60
Abstract · 27 pages, 0 figures

Laws and regulations deeply shape physicians' workflows and impact how they incorporate AI-based decision support tools to care for patients. Despite calls to comply with legal standards, many AI-based decision support tools have not transitioned well to clinical settings. The medical community remains concerned about their legal liability, risks to patients, and limited clinical value of such tools. To better understand how to address these concerns in the design of AI tools for clinical decision-making, we conducted 19 interviews with physicians in the U.S. We used scenario-based design to investigate physicians' legal concerns in the context of an AI-based drug recommendation tool. Drawing on the ethics of care, we surface connections between physicians' care and the healthcare laws and regulations that deeply shape their information needs. We describe how physicians wanted to use the tool in individual and collective practices to care for themselves, their patients, their colleagues, work institution, and even the broader medical profession. We discuss how the design of AI-based decision support tools can present legal ambiguities that would prevent their caring actions and lead to non-use of the AI tool. Through our work, we contribute to responsible AI development in healthcare. Our analysis illuminates why and how certain kinds of information reduce legal ambiguity and support physicians' care when using AI-based decision support tools. We contribute an understanding of how to help physicians navigate legal liability and draw implications for grounding AI tools' designs in their care.

0.37