CS Paper Radar

Communicating with the Entire Population: Machine Translation of Emergency Alerts and Warnings

Jen Wilson, Vipasha Bansal, William D. Lewis

cs.CL ·submitted 2026-10-23 16:17 UTC · v1
survey

Abstract

It is essential that we communicate with the entirety of an affected population during a crisis, including those who do not speak a region's dominant language(s). Commercial Translation tools offer a tempting solution to broaden access and can provide a lifeline when no other resources are available. However, Emergency response managers often operate in the dark regarding the quality and adequacy of these automated outputs. In this paper, we conduct a nationwide survey of emergency managers to understand their current workflows, perspectives, and operational challenges. We then evaluate commercial translation tools against these needs, shedding light on their utility and offering practical guidance on how and when these tools can be safely used in emergency response.

Comment
In Proceedings of the Language Technologies for Crisis Preparedness and Response (LT4CPR Workshop), Hengqin, Zhuhai, China. Association for Computational Linguistics. Nov 2026

Closest papers

Nearest neighbours by meaning across the whole index.

SIREN: Situation-aware Index for Response and Emergency ActioNability in Crisis Translation

cs.CL ·2026-10-12 ·Maria Carmen Staiano, Lifeng Han, Johanna Monti, Francesca Chiusaroli
code/data similarity 0.74
Abstract · accepted at The First Workshop on Language Technologies for Crisis Preparedness and Response (LT4CPR) at AACL-IJCNLP 2026

Machine translation (MT) is increasingly utilized in multilingual crisis communication, yet conventional evaluation does not directly assess whether translated emergency messages preserve their actionability. We introduce SIREN (Situation-aware Index for Response and Emergency ActioNability), a human evaluation framework assessing whether translation choices compromise protective action, directive force, and urgency of the source text. We evaluate SIREN in two complementary settings. At the error level, we apply it to HOPE-annotated Italian--English translations produced by ChatGPT-4o and Google Translate: 31.63\% and 48.73\% of errors, respectively, compromise actionability, predominantly by affecting the protective action. At the message level, we evaluate English--French translations produced by TranslateGemma-4B and Google Translate, with actionability preserved in 85.34\% and 92.67\% of translations, respectively. TranslateGemma-4B failures predominantly weaken directive force, whereas Google Translate failures more often compromise protective actions and receive Major impact ratings. These findings show that translation systems can differ not only in how often actionability is lost, but also in the nature and functional impact of these failures. Alongside SIREN, we contribute an English--French crisis-translation resource with human actionability annotations and post-edited reference translations, supporting further research on function-oriented evaluation for crisis communication. Our experimental data is available at https://github.com/mcstaiano/SIREN.

0.45

CrisiLangs: A Multilingual Resource for Crisis Warning Generation

cs.CL ·2026-10-22 ·Giacomo Gonella, Stefano Menini, Marco Guerini
similarity 0.70
Abstract

Communicating with the public during an emergency entails issuing warning messages promptly. In Europe, it often requires producing them in more than one language. Recent work has shown that language models can be trained to generate warning messages as a crisis unfolds, but the available data covers only English. To fill this gap, we present CrisiLangs, a multilingual extension of CrisiText (an English dataset of crisis events paired with warning messages written according to expert communication guidelines) to Italian, Bulgarian, German, and Greek. Expert native speakers translated the warning messages of 1,000 events drawn from 411 scenarios, for a total of 5,000 event-message pairs across five languages. We test the resource to fine-tune a multilingual model on all five languages jointly, with and without the emergency guidelines in the prompt, and compare it with a zero-shot baseline. The fine-tuned models outperform the baseline on ROUGE-L, BLEU and BERTScore in every language and configuration. In a human preference study, low decoding temperatures receive the most preferences in all five languages and in both configurations, while the effect of the guidelines varies by language.

0.23

Generative AI translations in high-stakes emergency messaging

cs.CL ·2026-10-06 ·Nune Ayvazan, Anthony Pym, Yu Hao
similarity 0.70
Abstract

Emergency messaging such as extreme-weather reports and earthquake instructions can involve high stakes, to the extent that translation errors can lead to tragic consequences. The use of machine translation or generative artificial intelligence might therefore not be recommended. On the other hand, time savings in the initial translation can allow greater investments of resources in revision and authorization processes, as well as a wider range of target languages. An experiment with generative AI translations of an earthquake instruction text from English into Chinese and Spanish shows that use of discourse-specific prompts can considerably improve understandability and actionability, although the translations may still not be trusted by translators. Human revision is still required, not only to detect errors but also because of the ethical need for someone to take responsibility for any errors or delays in such messaging.

0.23

Multilingual Agent System for Inclusive Wildfire Evacuation Guidance

cs.CY cs.AI ·2026-09-03 ·Shruti Kulkarni, Lynn Tong, Aditi Namboodiripad, Chelyah Miller, Helen Lin +3 more
published similarity 0.62
Abstract · This one is under submission (IEEE SpatialConnect Workshop2026)

Wildfire seasons have become 84 days longer in the current days than in the 1970s, causing enormous threats to one's financial status and short- and long-term health. During the fire, public agencies send out emergency messages to provide warnings and orders. Although 26 million people in the US have limited English proficiency, over 80% of those messages are only delivered in English, which can cause disproportionate information distribution and awareness. In order to better serve marginalized communities during emergencies, the authors developed BEACON, a service that provides comprehensive and personalized evacuation guidance, including navigation routes, personalized checklists, and a chatbot in the language that a user uses. Our current system ingests data including fire perimeter information, evacuation order status, and shelter information from Watch Duty. When a user is within a certain proximity from the fire, the system utilizes real-time GPS locations and nearby weather data from the National Oceanic and Atmospheric Administration (NOAA) to predict fire danger levels. The assessment model refreshment are dynamically scheduled based on fire progress and trends using XGBoost. If the location has a likelihood of fire danger, the system sends alerts with evacuation routes outputted from a polygon-avoidant routing pipeline. The application provides a context-aware multilingual agent that users can communicate with and is tightly connected to other features of the application. In addition, based on data that the user entered, the system dynamically generates and checks off personalized reminder items to provide an organized evacuation plan. The system's user interface dynamically changes its language settings based on the language the user most recently used in either setting or chatbot conversation for all the application elements.

0.32

Auditable Emergency Triage for Maternal and Newborn Care in India

cs.CL cs.AIcs.CY ·2026-09-08 ·Shobhit Jagga, Aman Dalmia, Niharika Priyadarshini, Neelima Devadas, Amrita K Prasen +5 more
similarity 0.58
Abstract · First three authors contributed equally

At Noora Health, our nurses answer more than 50,000 medical queries per month on our WhatsApp-based service that provides caregivers with on-demand support. Their most time-critical task is emergency triage: deciding which queries need immediate in-person attention. To support them, we built a system that uses a large language model (LLM) to classify whether a message is an emergency and provide a rationale for interpretability. But the system was opaque: analyzing mistakes meant reading reasoning chains for each message, which is infeasible at our scale. Prompt changes meant re-running a full evaluation to prevent regressions, which was both costly and operationally challenging. Clinicians follow a decision tree to make this call, but it was never documented or passed to the model, which relied on a flat list of danger signs. To address these issues, we decomposed triage into two steps: an LLM extracts canonical symptoms and patient context from the query using a clinician-authored vocabulary, and a deterministic rule engine captures the scenarios that indicate an emergency. We show that the new system raised recall from 0.565 to 0.810 and F1 from 0.606 to 0.702, with structured rules driving most of the accuracy gains while the decomposition provides auditability: clinical experts can inspect each stage of the new system to see whether the query was mistranslated, symptoms were incorrectly extracted, patient context was wrongly inferred, or the necessary rules were missing. They can add new rules independently without causing regressions and avoid running costly evaluations. Since deployment, the new system has triaged 152,421 patient queries and flagged 28,535 (18.7%) as emergencies. The over-escalation rate has been 17.8%, without any increase in missed emergencies. Clinicians have also added 48 new rules since deployment, evidence of the faster correction loop we set out to build.

0.16

From Triage to Discharge: A Survey of NLP Tasks, Methods, and Open Challenges in the Emergency Department

cs.CL ·2026-08-23 ·Dipankar Srirag, Aditya Joshi, Salil Kanhere, Padmanesan Narasimhan
top venuesurvey similarity 0.56
Abstract · Accepted to the EMNLP 2026 Main Conference; camera-ready version

Emergency departments (EDs) operate under time pressure, generating multimodal data such as clinical conversations, triage notes, and discharge documents. Recent advances in natural language processing (NLP), particularly pretrained transformers and large language models, have created new opportunities to support language and time-intensive stages of emergency care. Yet existing surveys map clinical NLP across the broader hospital workflow or focus on specific tasks. This survey analyses 46 papers spanning the three phases of ED: triage, diagnosis, and disposition, covering tasks such as triage classification, clinical summarisation, automatic diagnosis, report generation, and discharge documentation. We examine modelling paradigms, evaluation practices, and emerging benchmarks and shared tasks. Across tasks, we identify common trends, including a shift from task-specific neural architectures to pretrained language models, growing interest in interactive clinical systems, and increasing attention to clinically grounded evaluation. Finally, we detail open challenges such as limited generalisability, noisy clinical inputs, and workflow constraints that inform future ED-NLP research.

0.61

Refined Assessment for Translation Evaluation: Rethinking Machine Translation Evaluation in the Era of Human-Level Systems

cs.CL ·2026-10-24 ·Dmitry Popov, Vladislav Negodin, Ekaterina Enikeeva, Iana Matrosova, Nikolay Karpachev +1 more
top venuedatasetcode/datahighly cited similarity 0.56
Abstract · Published in Findings of EMNLP 2025. 17 pages. Dataset: https://huggingface.co/datasets/yandex/wmt24-en-ru-rate

As machine translation systems approach human-level quality, traditional evaluation methodologies struggle to detect subtle translation errors. We critically examine limitations in current gold-standard approaches (MQM and ESA), including inconsistencies from variable annotator expertise, excessive categorization complexity, coarse severity granularity, accuracy bias over fluency, and time constraints. To address this issue, we introduce a high-quality dataset consisting of human evaluations for English-Russian translations from WMT24, created by professional linguists. We show that expert assessments without time pressure yield substantially different results from standard evaluations. To enable consistent and rich annotation by these experts, we developed the RATE (Refined Assessment for Translation Evaluation) protocol. RATE provides a streamlined error taxonomy, expanded severity ratings, and multidimensional scoring balancing accuracy and fluency, facilitating deeper analysis of MT outputs. Our analysis, powered by this expert dataset, reveals that state-of-the-art MT systems may have surpassed human translations in accuracy while still lagging in fluency - a critical distinction obscured by existing accuracy-biased metrics. Our findings highlight that advancing MT evaluation requires not only better protocols but crucially, high-quality annotations from skilled linguists.

0.64

Locally Deployable Small Language Models for Emergency Department Decision Support: A Systematic Benchmark of Fine-Tuning Strategies

cs.CL cs.AI ·2026-08-10 ·Qingfeng Zhang, Yuanxiong Guo, Yanmin Gong
publishedbenchmarkcode/data similarity 0.56
Abstract · Accepted to AMIA 2026 Annual Symposium

Deploying large language models (LLMs) for decision support in emergency departments (EDs) faces two major challenges: privacy risks of transmitting patient data to closed-source commercial LLMs and the lack of systematic evaluation of fine-tuning strategies for locally deployable open-source small language models (SLMs). We benchmarked eight open-source SLMs using zero-shot prompting, prefix tuning, Low-Rank Adaptation (LoRA), and full fine-tuning on three ED tasks: triage level prediction, specialist referral recommendation, and diagnosis prediction. Using 2,083 MIMIC-IV-ED cases and Claude Haiku 4.5 and Claude Sonnet 4.5 as baselines, we found that LoRA fine-tuned open-source SLMs outperform commercial baselines on triage level prediction and specialist referral recommendation, while diagnosis prediction remains challenging for open-source SLMs. Confusion matrix analysis further shows that fine-tuned open-source SLMs can detect highest-severity patients missed by the commercial baselines. These results demonstrate that locally deployable SLMs can achieve clinically competitive performance for ED decision support.

0.35