CS Paper Radar

AI4Fire: Evaluating Large Language Models on Wildfire Tasks

Yue Zhao, Zuobin Xiong, Xiyang Hu, Zhangyu Wang, Xiongye Xiao, Ruolin Li

cs.CL cs.LG ·submitted 2026-10-07 21:52 UTC · v2
code/data

Abstract

Large language models (LLMs) are entering wildfire management, where overstated evaluations can cost property and lives. How do they perform on wildfire tasks, with and without grounding? Bare means a model receives the task input alone. Grounded means it also receives one task-specific addition: for smoke detection, a smoke-free reference frame from the same camera. AI4Fire runs six core models bare and grounded on five wildfire tasks, zero-shot; a sweep adds 29 more. Our literature search on fire tasks found 138 works; none combines this roster, task coverage, and paired bare and grounded runs. We report three findings. (1) Grounding helped most where the addition carried the answer: a read-only SQL tool lifted every core model's database accuracy from at most 16 to at least 88 percent. (2) Simple rules were hard to beat: no core model outperformed repeating today's staffing count, and two open-weight models mostly copied the median of similar earlier fire-days, a worse forecast. (3) Public releases carry hazards: a fire-danger column separates the holdout perfectly, and 67 aerial fire frames carry smoldering or fire-free labels read from a clipped thermal maximum. We release prompts, responses, scores, code, and the survey record.

Comment
51 pages. Code: https://github.com/yzhao062/AI4Fire

Closest papers

Nearest neighbours by meaning across the whole index.

WildfireSpreadBench: The Metric Decides the Model in Wildfire Spread Prediction

cs.LG cs.CV ·2026-08-29 ·Arin Gopakumar, Marco Pannozzo
published similarity 0.70
Abstract · NeurIPS 2026: Tackling Climate Change with Machine Learning Workshop. 12 pages, 2 figures, 5 tables

Machine learning is being increasingly used to predict where active wildfires will burn the following day, helping inform evacuation boundaries and containment lines. Most models are evaluated using Average Precision (AP), which summarizes performance across all decision thresholds, although acting on a forecast requires choosing one. We benchmarked five discriminative architectures and one generative model on WildfireSpreadTS using a shared evaluation pipeline and two input configurations. We found that model rankings varied depending on whether performance was measured by AP or by threshold-dependent metrics like F1 and IoU. At the default 0.5 threshold, the highest-AP model flagged 4 to 5 times the observed active-fire area and ranked fifth of six on F1 and IoU, and the most recall-heavy model flagged 16 to 23 times. The three models whose flagged area stayed close to the observed area had AP scores 24 to 37% lower. Across models, we identified three distinct prediction profiles: over-predicting, balanced, and under-predicting, which AP alone could not distinguish. Expanding the input from 7 to 23 channels changed AP by 0.03 on average, against a 0.21 to 0.24 spread across models. Ultimately, these results show that AP alone can favor models whose predictions are poorly suited for operational wildfire forecasting.

0.48

A Simulation-Grounded Agentic VLM Framework for Wildfire Monitoring and Reporting

cs.CV cs.GR ·2026-10-01 ·Duowen Chen, Yuchen Sun, Zhiqi Li, Yuxuan Liao, Sinan Wang +2 more
similarity 0.67
Abstract

Effective wildfire monitoring requires relating visual evidence to physical fire dynamics, yet real videos with synchronized physical annotations are scarce and high-fidelity 3D simulation is costly. We present a simulation-grounded vision-language model (VLM) framework that automatically converts 2D wildfire simulations into labeled video episodes. A fixed Blender mapping produces low-detail 3D proxies aligned with simulator terrain, fuel layout, fire activity, and wind cues; controllable video generation supplies richer appearance. The proxies are intermediate representations rather than finely rendered final scenes. Generated videos and simulator labels form reusable multimodal memory for a training-free multi-agent VLM system that retrieves reference episodes, reconciles visual and memory-based predictions, and produces structured wildfire reports. On held-out generated episodes, video memory achieves 51.5% exact four-tag accuracy, compared with 22.6% for direct VLM querying and 16-17% for text-only memory; the complete system achieves 77.3% accuracy on six simulator-derived report fields. Component ablations, cross-generator tests, and three real-UAV evaluations assess retrieval, reporting, generator changes, and observable monitoring tasks. The framework connects automatic simulation-to-proxy conversion with memory-based VLM reasoning under scarce real-world physical annotations.

0.20

Accelerating Wildfire Hazard Mapping Across Canada with an AI Surrogate Modelling Framework

cs.AI ·2026-10-16 ·Hager Radi Abdelwahed, Toluwaleke Olutayo, Chirag Goel, Jana Osea, Aldo Zaimi +5 more
similarity 0.63
Abstract · Preprint - under review

Canada's increasing wildfire activity requires frequent national hazard mapping to guide proactive mitigation, including fuel management, land-use planning, infrastructure protection, and evacuation routing. However, process-based wildfire simulation frameworks (e.g., BurnP3 and BurnP3+) rely on computationally intensive Monte Carlo simulations to model burn probability, fire intensity, and rate of spread. These workflows can take days to weeks to process a single regional modelling unit. To overcome these computational bottlenecks, we present three contributions developed through collaboration between AI researchers and wildfire modelling experts: (a) a standardized framework for training, fine-tuning, and evaluating AI surrogate models against process-based wildfire simulators; (b) a multi-task deep learning surrogate trained on national-scale data to emulate burn probability, mean fire intensity, and mean rate of spread at a fraction of the computational cost (seconds); and (c) domain-specific metrics and counterfactual evaluation protocols to probe out-of-distribution generalization. Integrating geospatial data (fuel, terrain, ignition likelihood) and regional weather, our model approximates BurnP3+ simulation outputs, achieving strong performance across agreement, rank-correlation, error, and spatial-overlap metrics. It preserves downstream hazard classifications and produces physically plausible responses to expert-informed counterfactual interventions. Furthermore, we demonstrate zero-shot transfer to a separate regional BurnP3+ study and show that lightweight fine-tuning can substantially improve regional calibration and predictive agreement. On a representative region, the surrogate achieves 806-fold end-to-end CPU acceleration and uses 13% of BurnP3+'s host memory, facilitating rapid scenario analysis.

0.08

SAFIRE: Safety-Critical Benchmark for Fine-grained Fire and Smoke Understanding in Multimodal LLMs

cs.CV cs.AIcs.CL ·2026-09-07 ·Pengfei Li, Naufal Suryanto, Sicheng Zhang, Mohammad Alsharid, Muzammal Naseer
top venuebenchmarkcode/data similarity 0.63
Abstract · Accepted at EMNLP 2026 Findings. Project page: https://risys-lab.github.io/SAFIRE/ | Github: https://github.com/RISys-Lab/SAFIRE

Multimodal Large Language Models (MLLMs) show strong progress on vision-language tasks, yet their reliability in safety-critical settings remains underexplored. Fire-smoke understanding is central to public safety and disaster response, but most existing benchmarks lack diverse real-world scenarios and context-aware evaluation. We introduce SAFIRE, a large-scale benchmark for fire-smoke understanding in MLLMs, comprising 83K captioned images from 20 scenarios and 193K multiple-choice VQA (MCVQA) generated from a 9.7K-image subset, spanning 10 evaluation dimensions from basic perception to higher-order reasoning. A GPT-5.4-assisted multi-stage verification pipeline with MLLM majority voting ensures annotation quality. Evaluating ten open-source MLLMs (8B-38B) yields an average accuracy of 61.9%, exposing major gaps in safety-critical reasoning. We further show that adapting vision encoders with only 7% of our domain-specific data boosts fire-scene classification accuracy from 20.1% to 64.5%, indicating that carefully curated data can yield substantial gains even when data volume is limited. All datasets, models, and code are available at https://risys-lab.github.io/SAFIRE/.

0.41

From Wildfire Severity to Snow Persistence: A Multisource GeoAI Study of the 2020 Creek Fire

eess.IV physics.geo-phstat.AP ·2026-09-28 ·Parastoo Farajpoor, Mohammadreza Narimani
publishedcode/data similarity 0.63
Abstract · 16 pages, 9 figures, 5 tables. Preprint submitted to Frontiers in Forests and Global Change. Code: https://github.com/MohammadrezaNarimaniUC…

Accurate prediction of post-fire snow conditions does not necessarily establish how wildfire changed those conditions. We developed an explainable geospatial artificial intelligence framework combining multisource Earth observations, meteorological information, and matched before-after comparisons to examine seasonal snow persistence following the 2020 Creek Fire in California's Sierra Nevada. Harmonized Landsat Sentinel-2 observations were used to estimate the fraction of clear-sky observations containing snow during October-July for eleven water years, 2016-2026, on a 500 m analysis grid. We matched 3,778 snow-zone cells inside the fire perimeter to comparable unburned controls using terrain and pre-fire snow conditions; absolute standardized mean differences after matching were no greater than 0.047. HLS persistence agreed closely with MODIS, with a mean annual spatial correlation of 0.947. The landscape-average before-after control-impact contrast was +0.0017, with a year-level 95% confidence interval of -0.023 to +0.027. A stronger response emerged in the highest burn-severity class, where observed persistence increased by 0.026 relative to matched controls, equivalent to 2.6 percentage points. Under 5 km spatial cross-validation, XGBoost predicted raw post-fire persistence with an out-of-fold coefficient of determination of 0.811, whereas predictive skill for the fire-adjusted anomaly reached 0.046. Elevation and temperature together accounted for 65.7% of mean absolute model attribution. These findings distinguish a severity-associated optical snow response from the terrain-climate relationships that dominate predictive skill. Combining matched comparisons with explainable GeoAI provides a practical framework for forest monitoring that separates accurate environmental mapping from inference about disturbance effects.

0.71

Modular Deep Learning Mechanisms for Auditable Next-Day Wildfire Spread Prediction

cs.LG ·2026-09-15 ·Miguel Esparza, Aydin Ayanzadeh, Ahmad Mousavi, Ali Mostafavi
benchmark similarity 0.62
Abstract

Next-day wildfire prediction requires models whose forecasts can be evaluated alongside the assumptions and historical evidence used in their computation. Although deep learning can learn spatial patterns from remote-sensing data, predictive performance alone does not establish physical fidelity or operational trustworthiness. This study investigates three modular augmentations for next-day active-fire prediction: wind- and slope-conditioned attention biases, physics-feature retrieval-augmented output correction, and fire conditioned dual-stream gating. The attention biases expose prescribed directional preferences, while the retrieval module selects historical tiles using a nine-dimensional environmental and fire-state descriptor and applies a learned correction to a frozen model's logits. The modules are evaluated across five backbones on the Next Day Wildfire Spread benchmark, using staged ablations, directional audits, retrieval perturbations, calibration measures, and computational comparisons. The three-seed mean F1 score and area under the precision--recall curve (AUC-PR) of a SwinUNETR model with all three augmentations are 0.4216 and 0.3673. Then, a mixed ensemble (two augmented architectures and one non-augmented architecture) model achieves 0.4292 and 0.3790. Benefits vary across architectures, and retrieval-related improvements in AUC-PR do not consistently translate into higher F1. The constructed wind bias aligns closely with input wind, but its alignment with observed next-day fire displacement is much weaker, distinguishing prior inspectability from predictive physical fidelity. The study contributes a framework for exposing and evaluating selected domain-informed components within wildfire prediction models. Together, the results presented show that predictive performance, operational trustworthiness, and computational practicality need not be competing objectives.

0.41

Distilling Vision-Language Models for On-Device Fire Understanding

cs.AI ·2026-09-05 ·Mohammad Kazzazi, Zixuan Liu, Siavash Khajavi
similarity 0.62
Abstract

Vision-language models (VLMs) offer a promising alternative to conventional fire detection systems by reasoning about the semantic context of a scene and thus reducing false alarms, yet their large model size makes deployment on embedded fire sensors impractical. In this paper, we study how domain-specialized VLMs can be compressed for fully on-device deployment without losing the safety-critical behavior required for fire detection. We develop a teacher-student knowledge distillation framework in which large VLMs fine-tuned for fire understanding can be distilled into lightweight students. Experiments across multiple VLM families and model scales show that compact students preserve most of their teachers' fire-understanding capability. We further deploy the distilled models on our commercial Detectium fire detection sensor and jointly evaluate reasoning accuracy, latency, and memory usage. The results show that compression and deployment affect not only accuracy but also model failure modes, with Qwen2.5-0.5B providing the strongest overall deployment trade-off. Our findings provide broader guidance for deploying domain-specialized VLMs in resource-constrained, safety-critical settings.

0.25

Vision Language Models for Primary Structure Threat Identification in Wildland Urban Interface Fires

cs.CV ·2026-11-07 ·Zhangding Liu, Neda Mohammadi, John E. Taylor
published similarity 0.61
Abstract · 6 pages, 3 figures, 1 table. Accepted at the 2026 IEEE International Smart Cities Conference (ISC2), Porto, Portugal

Wildland urban interface (WUI) fires increasingly threaten communities and infrastructure, motivating the need for rapid visual assessment. Recent research has focused primarily on detecting fire and assessing post-event damage. However, structure-level fire threat identification during active WUI fire events remains underexplored. This study evaluates Qwen3-VL-8B-Instruct for this active-fire decision using two separately scored tracks: 1,824 synthetic images from 491 designed WUI scenarios and 45 frames from a controlled-burn experiment conducted in collaboration with Columbus Fire & EMS Department. Under zero-shot prompting, the model returned valid target fields for 1,808 synthetic images and 39 controlled-burn frames. Accuracy among valid outputs was 94.0% for the synthetic track and 69.2% for the controlled-burn track, compared with a 16.0% random-selection baseline for the latter. On the synthetic track, the model generally identified the intended target across varied designed WUI scenarios when the relevant fire-spread cues were visually distinct. Controlled-burn performance was well above random selection, but errors showed difficulty interpreting spatial relations in complex scenes. The dual-track protocol provides scalable stress testing in designed scenarios and a controlled-burn reference for assessing model behavior under physical fire conditions. Future work can use the synthetic data for VLM post-training and then evaluate the adapted models on larger and more diverse real WUI fire datasets.

0.28