CS Paper Radar

Shortcut Learning in Geospatial Foundation Model Embeddings for Cross-Disaster Damage Assessment

Ali Shibli, Andrea Nascetti, Yifang Ban

cs.CV cs.LG ·submitted 2026-10-09 15:50 UTC · v1
publishedcode/data

Abstract

Geospatial foundation models can support disaster damage assessment, but high accuracy on familiar disasters may hide shortcut learning: prediction from event or scene characteristics rather than evidence that transfers to a new disaster. We study embeddings from TerraMind, Prithvi-EO-2.0, DOFA, and DINOv3 on the multi-disaster datasets xView2 and BRIGHT. Holding out an entire disaster, instead of randomly splitting image tiles, reduces mean balanced accuracy from 0.807 to 0.638 on xView2 and from 0.818 to 0.568 on BRIGHT. Four sets of tests examine what information supports prediction: event and acquisition context, before/after image inputs, building count, and correct image pairing. Pre-event images remain predictive, and a classifier given only the annotated number of buildings reaches balanced accuracy of 0.629 and 0.653. Combining before/after features performs best on average on xView2, but does not improve over pre-only features on BRIGHT. Shuffling image pairs within the same disaster and damage class causes modest losses. Motivated by these findings, we test whether feature transformations learned only from training disasters improve generalization to unseen disasters. These transformations improve performance, with supervised contrastive learning performing best on xView2 and PCA on BRIGHT. Our results highlight the importance of evaluation on unseen domains for assessing model robustness and show that simple feature-level transformations can improve generalization. Code and experiments can be found at https://github.com/alishibli97/gfm-shortcut-learning.

Comment
NeurIPS REO Workshop 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Neighbourhood shortcuts in building damage assessment from very-high-resolution satellite imagery: diagnosis and a building-evidence plug-in

cs.CV ·2026-10-28 ·JianChong Guo, YuTing Wan, Kefan Li, AiLong Ma, YanFei Zhong
code/data similarity 0.71
Abstract · 22 pages, 9 figures, 11 tables. Code: https://github.com/SummerTide/neighbourhood-shortcut-bda

Building damage assessment (BDA) from very-high-resolution imagery supports disaster response, and deep models now achieve high overall accuracy on public benchmarks. Because damage is spatially clustered, most buildings share the damage level of their neighbours, so overall scores are dominated by such buildings, whereas damaged buildings among intact ones are rare and easily overlooked. A high overall score therefore cannot tell whether a model judges the building itself or relies on its neighbourhood as a shortcut, and this study tests which of the two current models do. We stratify buildings by their agreement with their neighbours, move buildings into new neighbourhoods with their pixels unchanged, and diagnose nine models covering the main paradigms of BDA. All models miss a third to half of the isolated damaged buildings, and in non-flood events they call 73--87% of intact buildings damaged once these are moved into a damaged neighbourhood. We propose a building-evidence plug-in that adds two experts to an existing model without retraining it. One expert sees only the building, and the other sees its surroundings but is trained with seamlessly swapped neighbourhoods so that it learns not to trust them. With each of the nine models, the plug-in maintains or improves overall accuracy, reduces errors on buildings that disagree with their neighbours by 8--18%, and cuts by 40--69% the intact buildings that a damaged neighbourhood turns into damaged ones. It also raises the detection of synthetic isolated damage from 1--6% to 13--39% and carries over to optical--SAR data and to an unseen earthquake. High overall scores thus conceal a systematic reliance on the neighbourhood, which building-level evidence reduces at low cost.

0.27

Revisiting the Task of Building Damage Assessment by Satellite Imaging

cs.CV ·2026-10-09 ·Anthony Cioppa, Virgile Blua, Sébastien Piérard, Pietro Ceccato, Marc Van Droogenbroeck
hot direction similarity 0.70
Abstract · 82 pages, 2 figures, 9 tables, 160 references

Natural disasters such as earthquakes, tropical storms, and floods cause extensive damage to buildings, and rapid, reliable damage maps are essential to guide emergency response. Automating this task, referred to as building damage assessment (BDA), from satellite imagery, has become an active research topic at the intersection of remote sensing, computer vision, and machine learning, particularly since supervised deep learning methods were shown to outperform earlier handcrafted approaches. Yet, despite a decade of intense activity and the release of several large-scale datasets, the field lacks a shared foundation. This chapter revisits the task of BDA from first principles, organized around three pillars: data, evaluation, and methods. We first clarify the candidate task for- mulations and argue for a sensor-agnostic, instance-based definition that tolerates missing pre-event imagery. We then review existing datasets along the axes of sensing modality, temporal configuration, and geographic coverage. On the evaluation side, we show that prevailing practices, largely based on a single aggregate score, are theoretically fragile, and we lay the foundations of a more informative, ranking-based, multi-score protocol. We further review methods ranging from task-specific deep networks to recent Earth observation foundation models, noticing that generalization to unseen events and regions remains an open difficulty. Finally, we discuss how these elements can inform the organization of a BDA challenge, including cross-validation, challenge, and private data-sharing setups designed to build trust between developers and users.

0.31

Wildfire-SSR: Geolocation-Aware Multimodal Fusion for Structure-Level Wildfire Risk Assessment

cs.CV ·2026-11-16 ·Yu-Hsuan Ho, Yiming Xiao, Hao-En Lai, Ali Mostafavi, Samuel D. Brody
similarity 0.66
Abstract

Structure-level wildfire vulnerability assessment is critical for reducing losses in wildland-urban interface communities, yet existing methods often fail to combine parcel-scale ignition pathways with community-scale geographic coverage. This study introduces Wildfire-SSR (Satellite and Street-view for wildfire Risk), a geolocation-aware multimodal framework that integrates macro-scale satellite context with micro-scale street-view evidence. A Vision-Language Model extracts wildfire-relevant building and site attributes from pre-event street-view panoramas and converts them into structured semantic text. These features are spatially aligned with satellite imagery through a decoupled cross-attention fusion decoder, while a frozen satellite-pretrained DINOv3 encoder captures overhead visual context. Alternative geolocation embedding strategies are evaluated to anchor text-derived features within each satellite patch. The framework is tested on approximately 37,000 structures affected by the 2025 Eaton and Palisades fires in Los Angeles, using wildfire risk labels derived from relative post-event damage patterns to better represent inherent structural vulnerability. Rotary Positional Embedding provides the strongest spatial grounding, achieving a building localization F1 score of 0.6735 and a harmonic mean F1 score of 0.2476 for five-class pixel-level risk classification, outperforming unimodal baselines and an unanchored multimodal configuration. The model also shows a conservative bias in higher-risk classes, reducing severe-risk underestimation, and completes end-to-end assessment of the study area in 37 minutes on a single GPU. Wildfire-SSR reframes cross-view fusion as localized semantic alignment and provides a scalable tool for mitigation prioritization, resilience planning, and insurance-oriented wildfire risk assessment.

0.12

Towards Label-Efficient Geospatial Foundation Model Adaptation via Positive-Unlabeled Learning for Building Damage Mapping

cs.CV cs.AI ·2026-11-05 ·Hengwei Zhao, Qirui Wang, Zhuo Zheng, Xiaoyan Lu, Rusty Feagin +1 more
benchmark similarity 0.66
Abstract

Geospatial foundation models (GFMs) are reshaping remote sensing by shifting the field from task-specific modeling toward general-purpose geospatial intelligence. Mapping the damage to buildings stands to benefit substantially from this shift, as GFMs pretrained on large-scale Earth observation data provide generalizable geospatial representations that can facilitate effective responses to sudden-onset disasters. Although parameter-efficient fine-tuning (PEFT) facilitates GFM adaptation, parameter efficiency does not necessarily translate into label efficiency. Building damage mapping remains challenging due to scarce pre-existing labeled data or limited time for annotation. When weak supervision is introduced to alleviate the labeling bottleneck, we identify the difficulty of learning sufficiently discriminative representations as a key challenge for label-efficient GFM adaptation. To address this challenge, we propose a Positive-Unlabeled GFM Adaptation (PUMA) framework that requires annotations for only a small number of damaged buildings. More specifically, PUMA integrates non-contrastive weakly supervised representation refinement to mitigate the impact of noisy supervision on representation learning from a gradient perspective, and latent-state-based label disambiguation to enable reversible supervision refinement without requiring class prior estimation. Extensive experiments on disaster benchmark datasets from around the world demonstrated the effectiveness and robustness of PUMA. A real-world application to the 2025 Los Angeles wildfires further highlighted its potential to accurately map damage under highly limited supervision. More broadly, these findings suggest that weakly supervised PEFT offers a label-efficient pathway from GFMs to real-world applications.

0.15

Key Findings for Small UAS for Post-Disaster Damage Assessment from the CURSE'26 Exercise

cs.CV ·2026-10-21 ·Thomas Manzini, Stephen Johnson, Priyankari Perali, David Merrick, Robin Murphy
similarity 0.65
Abstract · 8 pages, 5 figures, 2 tables

The CLARKE building and road damage assessment AI classifier was incorporated into the Catastrophic Unmanned Remote Sensing Exercise (CURSE) 2026, which trained 23 small uncrewed aerial systems (sUAS) teams from state and local agencies and non-governmental organizations in Florida and Missouri on how to collect imagery for an incident command Remote Sensing Cell. Although CLARKE has been operationally deployed to two hurricanes (Debby and Helene), CURSE'26 presented an opportunity to evaluate the machine learning model with a larger set of never-before-seen orthomosaics. The results show that building and road damage assessment together ran in 3.5 minutes on average, and that building damage assessment achieved a Micro F1 of 0.834 with no manual adjustments to correct spatial misalignment and delivered that performance in both Florida and Missouri, demonstrating good geographic transfer. It also showed that road damage assessment performed poorly due to sensitivity to misalignment between road maps and the images, and to lighting and environmental conditions in an urban setting. The analysis of CURSE'26 presents five other findings of value to the sUAS community: that the use of sUAS and machine learning is now operationally normalized and accepted, sUAS teams in the field can improve classification accuracy by timing their flights, exercises are good operational tests but their value for evaluating learning models is less clear, edge computing is needed to overcome the wireless upload barrier, and the application of sUAS and AI to disaster response has ethical and legal ramifications for pilots and data managers.

0.20

Keep Your Friends Close, and the Right Neighbours Closer: Disaster-Conditioned Kernel-Regularized Graph Attention for Building Damage Classification

cs.CV cs.LG ·2026-08-20 ·Fuad Hasan, Chul Min Yeum
top venue similarity 0.65
Abstract · Accepted in ECCV 2026

Disaster damage is spatial: buildings rarely fail in isolation. Yet using spatial context for damage classification remains surprisingly underexplored, and many pipelines still rely primarily on per-building appearance cues even when the dominant uncertainty is spatially structured. Complicating matters, the right neighbourhood is not the same across events. Floods, hurricanes, and wildfires can exhibit very different clustering behaviour, making spatial reasoning valuable but easy to misuse - naive context aggregation can improve visual coherence while oversmoothing boundaries or propagating structured errors. We study this tension on xBD (the dataset used in the xView2 challenge) in a controlled post-localization, classification-only setup: each building is represented by a pre/post combined (PPC) patch cropped from the provided polygons, and spatial context is modelled with GPS-derived building graphs. Our approach keeps local evidence "close" by preserving strong spatial relationships in disaster damage patterns, while bringing only the right neighbours "closer" through a disaster-type-conditioned graph model that injects a learnable multi-scale spatial kernel prior into attention, allowing the effective neighbourhood scale to adapt across disaster types rather than being learned as a single global smoothing rule. To discourage coherence-by-smoothing, we add a residual de-correlation loss that penalizes positive Moran's~I in prediction residuals. We evaluate the method under event and dataset shift with a leave-one-event-out (LOEO) protocol on xBD and cross-dataset transfer from xBD to Ida-BD. The model improves macro-F1 and substantially reduces residual spatial autocorrelation under zero-shot event shift, indicating better use of spatial context rather than naive smoothing and enabling more reliable transfer to unseen events within known disaster types.

0.66

GeoPhysAdapter: Scale-Matched Geophysical Adaptation for Cross-Domain Landslide Mapping with Vision Foundation Models

cs.AI cs.CV ·2026-08-10 ·Zhihang Liu, Mei-Po Kwan, Jinlin Wu, Hao Li
publishedcode/data similarity 0.64
Abstract

Newly triggered landslides rarely carry immediate annotations, so cross-domain transferability determines the value of landslide mapping for emergency response and regional risk assessment. Vision foundation models have strengthened representational transfer, yet on unseen regions, events, and data sources they still generate high-confidence false alarms. Terrain, material, and rainfall triggering can constrain such errors, but their supports are local, regional, and event-scale, so that resampling onto a 10~m grid misaligns them with the segmentation decision unit and compounds the uncertain geographic context problem (UGCoP). We propose GeoPhysAdapter, which anchors on a frozen vision foundation model, restricts terrain, material, and triggering to dense spatial guidance, regional modulation, and event-timing forcing, and applies bounded adaptation at two decision units, the pixel and the candidate landslide body, reverting exactly to the visual prediction where support is insufficient. On an event-isolated PILD dataset of four public sources, 55 global landslide events, and 7,890 test samples, 70.3% of cross-domain false-positive mass lies in near-pure spurious bodies of median equivalent diameter 207m, matching coarse-prior support rather than the pixel. Pixel-level adaptation removes a net 507,817 erroneous pixels and reduces error by 7.76%, whereas raising the decision unit to the candidate body, under identical samples, anchor, and baseline, increases error reduction to 23.99%, approximately 3.1 times the pixel-level effect, improves IoU by 0.031 (14.2% relative), and corrects 9.92 pixels per pixel harmed. The data and code are publicly available at: https://github.com/Liu-Zhihang/geophysadapter.

0.41

When Satellite and Street Views Disagree: Cross-View Complementarity in Disaster Damage Assessment

cs.CV ·2026-10-03 ·Yifan Yang, Lei Zou
published similarity 0.64
Abstract · 4 pages, 3 figures; Accepted for publication in the proceedings of the 5th ACM SIGSPATIAL International Workshop on Searching and Mining Lar…

Satellite and street-view imagery see different parts of a damaged building: roofs from above, facades and debris from the ground. Average accuracy hides whether fusion methods use this complementary evidence. We study the conflict cases, the samples on which independently trained single-view models disagree, as a diagnostic subset. On paired collections from the 2025 Eaton wildfire and Hurricanes Ian and Milton, conflicts make up 9.6-33.2% of the test data. Fusion improves conflict accuracy over either view, yet an oracle that selects the correct single view per conflict remains 0.19-0.32 above the best evaluated fusion method. A linear fusion gate over building-visibility features, calibrated confidences, and disagreement recovers part of this headroom where ground photographs are aimed at the building: on Eaton it beats end-to-end fusion by +0.072 conflict accuracy p < 10^-4 and calibrated averaging by +0.051 p = 0.0001; on the panoramic hurricane datasets it is indistinguishable from end-to-end fusion. Cropping panoramas toward the building raises the conflict gain of fusion, whereas random crops of the same size do not. The spatial density of conflicts, computed without labels, correlates with tile-level wildfire damage Spearman r = 0.615, 25 tiles. Conflict cases thus show how much complementary evidence fusion uses, which view it carries under each hazard and capture setting, and where current methods fall short.

0.48