CS Paper Radar

A Simulation-Grounded Agentic VLM Framework for Wildfire Monitoring and Reporting

Duowen Chen, Yuchen Sun, Zhiqi Li, Yuxuan Liao, Sinan Wang, Bart van Bloemen Waanders, Bo Zhu

cs.CV cs.GR ·submitted 2026-10-01 20:22 UTC · v1

Abstract

Effective wildfire monitoring requires relating visual evidence to physical fire dynamics, yet real videos with synchronized physical annotations are scarce and high-fidelity 3D simulation is costly. We present a simulation-grounded vision-language model (VLM) framework that automatically converts 2D wildfire simulations into labeled video episodes. A fixed Blender mapping produces low-detail 3D proxies aligned with simulator terrain, fuel layout, fire activity, and wind cues; controllable video generation supplies richer appearance. The proxies are intermediate representations rather than finely rendered final scenes. Generated videos and simulator labels form reusable multimodal memory for a training-free multi-agent VLM system that retrieves reference episodes, reconciles visual and memory-based predictions, and produces structured wildfire reports. On held-out generated episodes, video memory achieves 51.5% exact four-tag accuracy, compared with 22.6% for direct VLM querying and 16-17% for text-only memory; the complete system achieves 77.3% accuracy on six simulator-derived report fields. Component ablations, cross-generator tests, and three real-UAV evaluations assess retrieval, reporting, generator changes, and observable monitoring tasks. The framework connects automatic simulation-to-proxy conversion with memory-based VLM reasoning under scarce real-world physical annotations.

Closest papers

Nearest neighbours by meaning across the whole index.

WildFireGS: Physics-Based Wildfire Simulation in Large-Scale Semantics-Enriched Gaussian Splatting Forest Scenes

cs.GR ·2026-08-11 ·Nienke Driessen, Joris Rijsdijk, Sören Pirk, Wojtek Palubicki, Dominik L. Michels +1 more
similarity 0.68
Abstract

Climate-driven environmental change is driving an increase in both the frequency and severity of wildfire events, making accurate simulation and prediction critical for effective risk mitigation and landscape management. While recent physics-based wildfire models achieve high realism by explicitly simulating combustion, heat transfer, and fuel dynamics, they remain largely restricted to synthetic environments with complete and idealized knowledge of forest structure, limiting their applicability to real-world environments captured via aerial imagery. To provide a pathway toward real-world wildfire digital twins derived directly from observational data, we present WildFireGS, a physics-based wildfire simulation framework operating directly on large-scale, semantics-enriched 3D Gaussian Splatting forest reconstructions. Our approach bridges learning-based scene reconstruction and environmental simulation by augmenting Gaussian primitives with semantics and material properties that encode vegetation type and fuel characteristics. We introduce a particle-based combustion model that operates natively on Gaussian representations, simulating ignition, heat transfer, combustion, and flame propagation across complex forest structures. This enables direct physics-based simulation of fire behavior on reconstructed real-world environments, without requiring conversion to explicit meshes or volumetric grids. We demonstrate the modularity of WildFireGS through a rain-driven cooling mechanism in terms of an energy-sink process to realistically model fire containment. Evaluations on synthetic scenes and real aerial forest captures show physically consistent wildfire behavior, reproducing characteristic dynamics including propagation scaling with vegetation density, wind velocity, and terrain slope. In addition, we validate our model through novel firebreak experiments and biomass loss estimation.

0.04

AI4Fire: Evaluating Large Language Models on Wildfire Tasks

cs.CL cs.LG ·2026-10-07 ·Yue Zhao, Zuobin Xiong, Xiyang Hu, Zhangyu Wang, Xiongye Xiao +1 more
code/data similarity 0.67
Abstract · 51 pages. Code: https://github.com/yzhao062/AI4Fire

Large language models (LLMs) are entering wildfire management, where overstated evaluations can cost property and lives. How do they perform on wildfire tasks, with and without grounding? Bare means a model receives the task input alone. Grounded means it also receives one task-specific addition: for smoke detection, a smoke-free reference frame from the same camera. AI4Fire runs six core models bare and grounded on five wildfire tasks, zero-shot; a sweep adds 29 more. Our literature search on fire tasks found 138 works; none combines this roster, task coverage, and paired bare and grounded runs. We report three findings. (1) Grounding helped most where the addition carried the answer: a read-only SQL tool lifted every core model's database accuracy from at most 16 to at least 88 percent. (2) Simple rules were hard to beat: no core model outperformed repeating today's staffing count, and two open-weight models mostly copied the median of similar earlier fire-days, a worse forecast. (3) Public releases carry hazards: a fire-danger column separates the holdout perfectly, and 67 aerial fire frames carry smoldering or fire-free labels read from a clipped thermal maximum. We release prompts, responses, scores, code, and the survey record.

0.49

SAFIRE: Safety-Critical Benchmark for Fine-grained Fire and Smoke Understanding in Multimodal LLMs

cs.CV cs.AIcs.CL ·2026-09-07 ·Pengfei Li, Naufal Suryanto, Sicheng Zhang, Mohammad Alsharid, Muzammal Naseer
top venuebenchmarkcode/data similarity 0.66
Abstract · Accepted at EMNLP 2026 Findings. Project page: https://risys-lab.github.io/SAFIRE/ | Github: https://github.com/RISys-Lab/SAFIRE

Multimodal Large Language Models (MLLMs) show strong progress on vision-language tasks, yet their reliability in safety-critical settings remains underexplored. Fire-smoke understanding is central to public safety and disaster response, but most existing benchmarks lack diverse real-world scenarios and context-aware evaluation. We introduce SAFIRE, a large-scale benchmark for fire-smoke understanding in MLLMs, comprising 83K captioned images from 20 scenarios and 193K multiple-choice VQA (MCVQA) generated from a 9.7K-image subset, spanning 10 evaluation dimensions from basic perception to higher-order reasoning. A GPT-5.4-assisted multi-stage verification pipeline with MLLM majority voting ensures annotation quality. Evaluating ten open-source MLLMs (8B-38B) yields an average accuracy of 61.9%, exposing major gaps in safety-critical reasoning. We further show that adapting vision encoders with only 7% of our domain-specific data boosts fire-scene classification accuracy from 20.1% to 64.5%, indicating that carefully curated data can yield substantial gains even when data volume is limited. All datasets, models, and code are available at https://risys-lab.github.io/SAFIRE/.

0.41

FireWorldBench: Benchmarking Complex Physical World Intelligence through Coupled-Field Fire Dynamics

cs.AI ·2026-09-19 ·Qiang Chen, Hao Guo, Huatai Zhu, Tairan Huang, Yichao Cao +5 more
benchmark similarity 0.65
Abstract

Understanding the physical world requires more than object recognition, scene description, and short-term visual prediction, as real-world physical systems involve multiple continuous fields, latent causal mechanisms, partial observations, and intervention-sensitive dynamics. We propose FireWorldBench, a benchmark for evaluating complex physical world intelligence in multimodal large language models and agents through coupled-field fire dynamics. Fire provides a canonical stress-test environment, where multiple interacting physical fields jointly shape observable states and temporal dynamics. FireWorldBench is organized along two complementary axes, a physical capability axis and a fire scenario task axis, jointly covering physical-state understanding, temporal dynamics, causal mechanisms, and intervention reasoning. The benchmark comprises 520 fire-world entries, including 494 controlled simulation worlds and 26 real-world-aligned event groups, spanning 47 scene archetypes across 7 environment families. These entries combine structured textual observations, multiple 2D physical-field visualizations, and 3D event-level scene modeling, yielding 9,074 text-image interleaved question-answer pairs across choice-based and open-ended report-generation formats. FireWorldBench evaluates whether models can infer latent physical states, explain underlying mechanisms, forecast coupled-field evolution, and assess intervention consequences from multimodal partial observations, providing a challenging testbed for complex physical world intelligence.

0.30

Vision Language Models for Primary Structure Threat Identification in Wildland Urban Interface Fires

cs.CV ·2026-11-07 ·Zhangding Liu, Neda Mohammadi, John E. Taylor
published similarity 0.64
Abstract · 6 pages, 3 figures, 1 table. Accepted at the 2026 IEEE International Smart Cities Conference (ISC2), Porto, Portugal

Wildland urban interface (WUI) fires increasingly threaten communities and infrastructure, motivating the need for rapid visual assessment. Recent research has focused primarily on detecting fire and assessing post-event damage. However, structure-level fire threat identification during active WUI fire events remains underexplored. This study evaluates Qwen3-VL-8B-Instruct for this active-fire decision using two separately scored tracks: 1,824 synthetic images from 491 designed WUI scenarios and 45 frames from a controlled-burn experiment conducted in collaboration with Columbus Fire & EMS Department. Under zero-shot prompting, the model returned valid target fields for 1,808 synthetic images and 39 controlled-burn frames. Accuracy among valid outputs was 94.0% for the synthetic track and 69.2% for the controlled-burn track, compared with a 16.0% random-selection baseline for the latter. On the synthetic track, the model generally identified the intended target across varied designed WUI scenarios when the relevant fire-spread cues were visually distinct. Controlled-burn performance was well above random selection, but errors showed difficulty interpreting spatial relations in complex scenes. The dual-track protocol provides scalable stress testing in designed scenarios and a controlled-burn reference for assessing model behavior under physical fire conditions. Future work can use the synthetic data for VLM post-training and then evaluate the adapted models on larger and more diverse real WUI fire datasets.

0.28

Annual Earth-observation embeddings encode wildfire disturbance and support simplified burned area mapping

cs.CV ·2026-09-22 ·Jovana Knezevic, Clement Atzberger, Zhengpeng Feng, Adam F. A. Pellegrini, Srinivasan Keshav +1 more
benchmark similarity 0.63
Abstract

Medium-resolution (10-30 m) burned area mapping is vital for monitoring wildfires and their impacts, but remains difficult to scale. Existing methods require either curated fire-specific imagery or dense time-series analysis. Here, we tested whether annual Earth-observation embeddings retain wildfire disturbance signals sufficiently to map burned areas without either requirement. Using Tessera and AlphaEarth embeddings, we tested individual burn-scar delineation, mapping of all same-year fires within an area, regional wall-to-wall mapping, cross-continental transfer, and intra-annual fire timing. Tessera strongly encoded wildfire disturbance, allowing even linear models to separate burned from unburned pixels; the signal was weaker in AlphaEarth. Models trained on a single Tessera embedding matched or exceeded equivalent models using paired pre- and post-fire HLS imagery, and outperformed post-fire imagery alone. The same approach mapped all same-year fires within benchmark scenes (F1 = 0.90). Applied across California, with no California fire data used for downstream training, it recovered 97% of reference burned area and detected substantially more small and medium-sized fires than GABAM or MCD64A1. Separately, a model trained on 2018-2021 US fires transferred without retraining to 88 European fires from 2024-2025 (F1 = 0.88). For well-detected fires, ignition timing was recovered with a mean absolute error of 13 days. Performance declined for fires ignited near the end of the calendar year, and wall-to-wall deployment produced systematic false positives in some unseen landscapes. Annual embeddings nevertheless achieve high segmentation accuracy while moving the burden of dense time series processing upstream, providing a promising path towards simpler regional burned area mapping.

0.14

AgentVQA: A Unified Benchmark for Agentic Visual Understanding

cs.CV ·2026-10-22 ·Advait Gupta, Sachin Konan, Tom Yin, Gabriel Sarch, Zhuang Liu
benchmarkcode/data similarity 0.63
Abstract · Project page: https://zlab-princeton.github.io/AgentVQA

Vision-language models (VLMs) can perform a broad range of tasks across diverse settings, yet their performance in agentic contexts remains poorly understood. Although online evaluations in simulators and real-world environments are essential for measuring interactive agent performance, they are costly and challenging to scale. Complementary offline VQA benchmarks can enable faster and more reliable comparisons across agentic domains, yet existing benchmarks remain limited and fragmented. To address this gap, we introduce AgentVQA, an offline benchmark for evaluating VLMs on realistic agentic scenarios across five agent domains: Web Agents, Robotics, Egocentric Videos, Games, and Spatial Planning. We build AgentVQA by manually selecting 1000 challenging, real-world agent examples from 23 source datasets and adapting them for reliable offline evaluation. The resulting benchmark preserves realistic scenarios from interactive environments across diverse domains, and its model rankings track those from corresponding online evaluations. Across the 22 evaluated models, the best achieves only 58.5% accuracy. Our cross-domain analysis shows that errors are driven primarily by failures to ground and maintain accurate representations of the current and evolving visual state, including localizing relevant UI elements, interpreting motion, tracking state changes over time, and inferring 3D spatial relationships, rather than by task misinterpretation or downstream logical reasoning.

0.35

EnergyEminence: Source-Aware Environmental Calibration and Evaluation in a Physics-Grounded Grid Digital Twin

cs.LG eess.IV ·2026-09-28 ·Huy Trinh, Michael Mai, Yu Nong
similarity 0.63
Abstract

Power-grid digital twins must combine data-driven prediction with physically meaningful state evolution while preserving the provenance of environmental observations. This paper presents an early-stage EnergyEminence testbed that couples an IEEE 118-bus-style graph-temporal predictor, nonlinear AC cascade simulation, and operator-dashboard-like temporal replay. In addition, we introduce a shared bounded calibration that converts wildfire-detection confidence and spatial extent into source-comparable wildfire interpretable and explainable evidence. We then evaluate it with visually diverse fire and hard-negative videos. Sixteen synthetic environmental videos are curated to generate 160 source-tracked grid scenarios, and a source-video-disjoint test yields 10 true positives, 8 false positives, 22 true negatives, and no false negatives. The errors occur in stressed, non-cascading scenarios conditioned on an unseen growing-fire source. Our diagnostic then reveals environmental shortcut learning that is obscured by scenario-level random splitting. The paper therefore contributes a data-centric and inspectable evaluation workflow for multimodal grid-resilience models, together with evidence supporting separation of environmental alerting from electrical cascade inference

0.22