CS Paper Radar

Digital Medicines Information at National Scale: Search Behaviour and System Performance of MediVerify Across 1.5 Million Queries

Praveen Charuka Athauda-Arachchi, Pandula Mahesh Athauda-Arachchi, Rohini Fernandopulle

cs.CY ·submitted 2026-09-24 02:53 UTC · v1
published

Abstract

National-scale digital health platforms generate usage data that reveal population healthcare needs. MediVerify, Sri Lanka's online medicines information platform providing access to National Medicines Regulatory Authority (NMRA)-approved medicines, recorded over 1.49 million queries in its first year. Large-scale medicine search behaviour in low- and middle-income countries (LMICs) remains poorly characterised. Objectives: To characterise medicine information-seeking behaviour, identify mismatches between public demand and essential medicines policy, and evaluate technical performance. Methods: We retrospectively analysed 1,497,304 anonymised queries submitted between July 2024-2025. Queries were normalised and mapped to 11,933 approved medicines using fuzzy matching based on Levenshtein distance (threshold <=5). Therapeutic categories were assigned using an Anatomical Therapeutic Chemical (ATC)-aligned classification. Query patterns and system performance were analysed, and energy consumption was estimated from processing times. Results: Vitamins/minerals (11.74%), antibiotics (10.57%), anti-diabetes (7.02%), and antihypertensives (6.84%) dominated searches. The top 20 accounted for 37.3% of queries, while approximately 40%-50% of registered medicines were never queried. Zero-result queries (~ 1%) indicated unmet information needs. Frequently searched medicines diverged from essential medicines lists. Median latency was 8ms, with >99% query resolution. Estimated energy consumption was approximately 0.12 kWh per million queries. Conclusions: Large-scale medicine search data provide insights into healthcare information demand and policy alignment. MediVerify demonstrates that a national digital health platform can achieve high utilisation at low computational and environmental cost. Usage analytics could strengthen pharmaceutical policy and digital health infrastructure in LMICs.

Comment
10 pages, 6 figures, Published in Frontiers in Digital Health
Journal ref
https://www.frontiersin.org/journals/digital-health/articles/10.3389/fdgth.2026.1855835/full
DOI
10.3389/fdgth.2026.1855835

Closest papers

Nearest neighbours by meaning across the whole index.

An Offline Smartphone Check for Drug Interactions and Past Reactions in Handwritten Prescriptions: A Technical Feasibility Study in Rural Bangladeshi Outpatient Care

cs.HC cs.LG ·2026-08-27 ·Shahir Abdullah
benchmark similarity 0.55
Abstract · 10 pages

Rural Bangladeshi outpatient clinics keep no patient records, prescriptions are handwritten, and physicians have under a minute per patient, so they easily miss a known adverse drug reaction (ADR) or drug-drug interaction (DDI). We built and evaluated an offline Android application that photographs a handwritten prescription, detects and reads each medicine line, resolves brand names to active ingredients, and retrieves, but never predicts, documented interactions and the patient's recorded reactions. This quantitative technical-feasibility study, with qualitative error analysis, involved no patients or clinicians. Reference data combine two Bangladeshi drug registries (27,880 brands) with 190,421 DDI pairs. A YOLOv8n line detector reached mAP@0.5 = 0.687 on held-out prescriptions but recalled only 60.6% of lines at the deployed cutoff. On-device ML Kit read 11.8% of 780 benchmark handwritten words exactly. We trained a CRNN-CTC reader on public handwriting data, decoded it against the brand lexicon, and fine-tuned it on 660 author-labelled prescription crops. On benchmark words, the reader accepted the correct ingredient for 41.5% and a wrong one for 4.2%, versus 21.0% and 5.3% for ML Kit at its preset cutoff (McNemar p<10^-20). On scorable lines from held-out real prescriptions, the reader accepted the correct ingredient for 36.0% [26.8, 46.3] and a wrong one for 2.2% [0.6, 7.8]. In 62 simulated runs on a mid-range phone, median time per patient was 26-31 s and 59 of 62 runs finished within 60 s, but 62% of medicine lines still needed a manual check. A cross-visit medication history added after the evaluation passed seven automated tests and one two-visit functional test but was not evaluated. The pipeline is feasible within the clinic workflow; reading handwriting, not retrieval, is the bottleneck. Nothing here demonstrates a clinical benefit.

0.32

Who Anchors AI Overviews in Health? Baidu, Google, and the Geography of Authority

cs.IR cs.CY ·2026-09-06 ·Mingyue Zha, Ho-Chun Herbert Chang
similarity 0.55
Abstract

Artificial intelligence is being rapidly incorporated into traditional search systems, yet scant work audits the information disparities across platforms, geography, and languages. We address this gap by comparing Google and Baidu's AI Overview systems for health queries, and measure informational anchors that emerge. Auditing 1,920 health queries across 12 countries and 4 languages, we find that Google and Baidu exhibit vertical integration, routing users toward their own company platforms in AI Overviews rather than a diverse set of primary sources. Smaller, lower-localization countries receive fewer domestically sourced references for health queries. Issuing the same query in a country's official language rather than English raises the share of locally sourced citations approximately 3.5- to 13.5-fold. Comparing queries across health topics of varying severity and controversy, including Traditional Chinese Medicine as an example, we also show that health disclaimers are multidimensional and vary across language and culture. We discuss how generative search influences access to health information, and the urgent need for culturally-aware oversight of these systems that influence critical health decisions.

0.20

Understanding AI Provider Recommendations in Local Service Markets

cs.CY cs.CLcs.IR ·2026-09-16 ·Hazem Ibrahim, Yasir Zaki
similarity 0.53
Abstract · 12 pages, 6 figures

When someone asks an AI assistant which doctor to see or which firm to trust with their savings, the answer is a referral. We audit AI provider recommendations in four registry-backed service domains across the 100 largest U.S. metropolitan areas, matching every recommendation against the official registry for its domain (Medicare clinician and facility records, and SEC adviser disclosures), under three conditions: an open-weight model, a proprietary model without web search, and the same proprietary model with search. Without search, both models largely fabricate recommendations in the domains the web covers thinly. Only 4% of the open-weight model's recommended doctors and 11% of the proprietary model's match a clinician in the queried city, and the open-weight matches are name coincidences: its matched clinicians are no likelier to be primary-care doctors than names drawn at random from the registry. With search, 64-71% of recommendations in the same domains match a real provider. Search also changes who is recommended. Without it, recommended advisory firms carry SEC misconduct disclosures at 3.6 times the registry base rate, even after adjusting for firm size; with search, significantly below it. Restaurants, where quality and visibility are separately measurable, show a 3-5x review-count premium but a rating premium of at most a tenth of a star. Finally, search largely removes the metro-size penalty: without it, real recommendations concentrate in the largest metros; with it, match rates are similar across metro-size terciles. Whether an AI referral is trustworthy depends strongly on its retrieval configuration rather than on the underlying model alone, yet an answer produced without retrieval often carries no sign that its recommendations were never verified.

0.19

PlainPairs: A Corpus Pairing the Prescriber and Patient Texts of U.S. Prescription Medicine Labels

cs.CL cs.CY ·2026-11-16 ·Obada Kraishan
similarity 0.51
Abstract · 19 pages, 9 figures, 11 tables

A prescription medicine in the United States comes with labeling written for prescribers, and many medicines also carry a text written for patients: a Medication Guide or a patient package insert. The two texts describe the same medicine for two readers, yet public resources rarely hold them side by side. I present PlainPairs, an open corpus built from the openFDA drug label files. Version 0.1.0 holds 35,177 prescription labels that carry a patient text, 21,416 distinct patient texts, 250,829 patient-text segments matched to prescriber sections through their regulatory headings, reading-level measures on both sides, and rule-based counts of plain-language wording. A second source, the National Cancer Institute's cancer information summaries, adds 756 summaries in health-professional and patient versions, in English and Spanish. Measured on the structure of the label, the median Medication Guide reads at grade 8.7 by Flesch-Kincaid and at grade 11.1 by SMOG. The same guides measured on the flattened text that the bulk files serve read about two grades higher, a difference that comes from lost list and heading boundaries. Patient texts address the reader in 41 of every 100 sentences, against 0.2 in prescriber sections. A blind hand check of 100 pairs supports the heading-based alignment for seven of ten heading types. The corpus, its code and its documents are public, and the release keeps texts that show a company copyright mark out of the text tables.

0.27

A corpus-specific clinical RAG system matches or outperforms newer frontier LLMs on HealthBench

cs.CL cs.AIcs.HCcs.IR+1 ·2026-08-12 ·Praveen Reddy, Charuta Mandke, Suvrankar Datta, Sarah Khan, Siddharth Reddy Anthireddy +2 more
benchmark similarity 0.51
Abstract · 2 tables

General-purpose large language models (LLMs) have recently been reported to match or exceed specialized clinical AI tools on medical benchmarks, but such comparisons draw on a narrow set of systems and on benchmarks developed largely in high-income settings. We evaluate VITA, a retrieval-augmented generation (RAG) system purpose-built for contextual knowledge retrieval in India and other low- and middle-income (LMIC) settings. VITA retrieves from a curated corpus of disease-specific guidelines, India-specific antimicrobial resistance data, national formulary constraints, and resource-limited care protocols; its architecture and corpus are proprietary, but the benchmark, the physician-written rubrics, and our full response and scoring outputs are public for independent verification. On 4,023 English-language HealthBench questions (80.5% of the benchmark), scored with a GPT-4.1 judge, VITA ranked first with 51.9% of possible rubric points, ahead of GPT-5.4 (46.1%), o4-mini (44.3%), Gemini 3.1 Pro (42.6%), and Claude Sonnet 4.6 (37.3%), and scored highest on 45.4% of questions. To test robustness to newer models and judge lineage, a 500-question subset was re-run against current-generation models (GPT-5.5, Claude Opus 4.8, Gemini 3.5 Pro, Grok 4.3) and graded by a neutral open-weight judge (DeepSeek-V4-Pro) sharing no lineage with any system tested. Here the gap narrowed to parity: VITA and GPT-5.5 were statistically indistinguishable on mean per-question score, while VITA led on points-weighted score and won the most questions. VITA's advantages in accuracy and completeness persisted under the neutral judge; its communication scores were lower. These results indicate that a purpose-built clinical RAG system remains competitive with frontier LLMs on an open benchmark, consistent with corpus specificity as a design variable that improves grounding at some cost to communication polish.

0.04

Does the Citation Matter? Quality, Grounding, and Physician-Rubric Accuracy of Google's AI Overviews for Medical Information

cs.IR cs.CLcs.CYcs.HC ·2026-10-29 ·Sam Agnoli, Brigid Reilly, Lily Barna, Emőke-Ágnes Horvát
benchmarkcode/data similarity 0.50
Abstract · 12 pages, 5 figures, 10 tables. Code and data: https://github.com/SamAgnoli/AIOverviews

Google's AI Overviews (AIOs) are the first result users encounter when searching the web, shifting the work of selecting and evaluating sources from the user to Gemini. This may be especially consequential for health information, given that Google's traditional algorithm was designed to prioritize quality sources, and inaccurate information carries health risks. We evaluated 3,611 benchmark health questions, comparing each AIO against that question's top five traditional results on source selection and quality. We also evaluated the AIOs' statement grounding in their cited sources and how accurate AIOs were against physician-written rubrics. Overviews appeared for 96.4% of questions but drew on substantially different evidence than traditional search: only 44.4% of the top five results were cited by the corresponding AIO, which instead overrepresented commercial health publishers (e.g., WebMD, Healthline) at the expense of academic and government sources (e.g., Cleveland Clinic, National Health Service). AIO-cited pages scored lower than traditional results on four of six well-established credibility criteria. The grounding of AIO statements to their sources was poor: after correcting for classifier error, an estimated 56.7% of cited statements were supported by their cited page, and a further 20.8% of all statements were factual claims with no citation at all. When AIOs were graded against physicians' rubrics, they earned 38.2% of all available points (e.g., accuracy 54.2%, completeness 21.6%), and among the subset of AIOs whose physician rubric anticipated a specific inaccurate or potentially harmful claim, 22.2% of AIOs contained the harmful content. Notably, the accuracy of AIOs was unrelated to the grounding of their statements. We recommend AIO literacy interventions for users and holding AIOs at least to Google's own health-ranking standards.

0.33

Real-World Adoption, Use, and Governance of an LLM-Based Electronic Health Record Search and Synthesis Platform

cs.CY ·2026-11-05 ·Jiunn-Tyng Yeh, Michael Gao, Marshall Nichols, Anika Mohan, Tareq Aljurf +6 more
similarity 0.50
Abstract

We evaluated the longitudinal adoption and use of Scout, an electronic health record-integrated large language model platform that provides cited search and synthesis of patient records, during staged deployment across a multi-hospital academic health system. Over 11 months, 2,635 users completed 80,465 tasks. 1,768 clinical users initiated use within the first 29 days of system-wide launch, which included a required training. Daily initiation increased 21.2-fold at launch, from 2.9 to 61.0 users per day. Of the 495 users who initiated use during the launch week, 310 did so in the first two days. Use spanned more than 20 recorded specialty categories, five care-practitioner roles, and 46,485 distinct prompts involving retrieval, synthesis, extraction, evaluation, and drafting, with 752 tasks saved for reuse. Among care practitioners who completed the required course, 77.9% went on to use Scout, with a median of 1.7 days after course completion. The highest-intensity user quintile accounted for 59.0% of tasks and 61.9% of tokens. Exploratory prompt analysis showed use across retrieval, synthesis, extraction, evaluation, and note-drafting. These findings suggest that rapid, broad adoption requires clinician-led task design and evaluation, traceable governance proportional to reach, centralized capacity planning and user support, and training integrated with access.

0.19

AskGLP1: A Knowledge Graph LLM Agent for Exploring GLP-1 Experience from Online Patient Narratives

cs.CL cs.AIcs.SI ·2026-10-26 ·Zhaoqian Xue, Anay Somani, Siddhartha Singh, Junting Mu, Zhongyin Wang +8 more
code/dataunusual similarity 0.49
Abstract · 16 pages, 4 figures, 11 tables

GLP-1 medications are increasingly used for weight management, and many people share their experiences online. These narratives show why people take the drugs and what they experience, but they remain unstructured and disconnected from formal pharmacovigilance. We therefore present AskGLP1, a system that transforms online patient narratives into a knowledge graph (KG) and answers questions about medication experiences. Through an LLM-assisted pipeline grounded in the MedDRA terminology, AskGLP1 screens nearly 8.5 million Reddit posts for medication experiences and distills 163,135 of them, with 2,506 WebMD reviews, into a KG of 385,790 drug-side-effect and drug-purpose relations. We find that GLP-1 experiences are source- and drug-specific. Online narratives share the most common gastrointestinal symptoms with FAERS, but they foreground everyday burdens such as fatigue and food aversion, whereas FAERS reports more often include injection-site reactions and selected serious diagnoses such as pancreatitis. We further find that posts report less nausea and vomiting for tirzepatide than for semaglutide, more fatigue for Wegovy than Ozempic and for Zepbound than Mounjaro. Evaluations show that the agent answers with verifiable evidence and declines requests for medical advice. Our findings reveal gaps between what FAERS captures and what people report online, and they call for patient-centered approaches to pharmacovigilance. Code, prompts, and evaluation questions are available at https://github.com/ZhaoqianXue/AskGLP1.

0.42