CS Paper Radar

LoCA: Enabling Multi-Modality in Geospatial Location Encoders

Panagiotis Koromilas, Eleanna Vali, Diogenis Antonopoulos, Eleni Saka, Dimitris Christofis, Nikolaos Ioannis Bountos, Konstantin Klemmer, Ioannis Papoutsis, Yannis Panagakis, Mihalis A. Nicolaou

cs.CV cs.LG ·submitted 2026-10-23 17:01 UTC · v1
theory

Abstract

Location encoders map geographic coordinates to high-dimensional descriptors through contrastive alignment with paired Earth Observation imagery. We prove that this construction is structurally limited: any Lipschitz location encoder produces an output of intrinsic dimension at most 2, regardless of architecture or bandwidth, which forces the contrastive loss to strictly exceed its asymptotic optimum. Empirically, joint training also collapses the encoder's embedding covariance to low rank. Although dimension-limited, current location encoders sidestep this spectral collapse by training against a frozen image encoder, which we reinterpret as a covariance regularizer that supplies a high-rank target for the location encoder to fold onto. However, extending this approach to multimodal inputs is challenging, as location encoders should simultaneously map a single location to multiple, potentially incompatible, descriptors. We identify two requirements a regularizer must satisfy for location encoding in multimodal regimes: co-alignment across modalities, since adapters alone cannot align independently pretrained encoders; and high effective rank, since the location encoder inherits its target's spectrum. We propose LoCA (Location Contrastive Alignment), a two-stage pipeline that satisfies both. Stage 1 jointly trains Sentinel-1 and Sentinel-2 encoders with contrastive learning, producing a co-aligned, high-rank multimodal target. Stage 2 freezes the image encoders and trains a location and a tabular metadata encoder as projection adapters with multi-view contrastive learning. Against five state-of-the-art encoders on 17 tasks ranging from land cover and wildfire forecasting to geophysical, socio-economic, biodiversity and climate applications, LoCA achieves the most task wins, is the only model to outperform every other on a majority of tasks, and wins 68.8% of all per-task comparisons.

Closest papers

Nearest neighbours by meaning across the whole index.

X Marks the Spot: Diffusion for Image Geolocalization with Location Representations

cs.CV ·2026-11-09 ·Brian Wei, Chih-Chuan Hsu, Tianyi Gao, Daniel Cher, Eric Xing +2 more
similarity 0.71
Abstract · 24 pages, 9 figures

Image geolocalization predicts the location at which a photo was taken. Accurate geolocalization requires discriminating between both physically and semantically adjacent areas on Earth. However, prior diffusion-based approaches predict locations by generating coordinates, which are semantically unstable over space. We avoid this limitation by learning geolocalization behavior in the pretrained representation spaces of location encoders, which provide both positional and semantic understanding. To make this space usable for coordinate prediction, we introduce a Location Autoencoder (LAE), which pairs a frozen location encoder with a learned decoder mapping embeddings back to coordinates with kilometer-level accuracy worldwide. In the LAE space, we train DIG, a latent diffusion model that transports noise to location embeddings conditioned on image features. DIG consistently outperforms coordinate-based generative models and sets a new state of the art for parametric geolocalization on three benchmarks: OSV-5M, Im2GPS3k, and YFCC4k. Notably, DIG is the first generative model to outperform retrieval-based methods, and further improves retrieve-and-rerank systems when used as a prior. Code and models will be released.

0.17

What Does CLIP Learn for Regional Geolocalization? Probing Visual Cues and Scene Configuration After Adaptation

cs.AI cs.CEcs.CV ·2026-08-22 ·Changyu Lee, Yeonsoo Park, Abdullah Alfarrarjeh, Seon Ho Kim
similarity 0.65
Abstract · 10 pages, 4 figures, 9 tables. Manuscript under review

Large collections of street-view imagery provide rich visual information about urban environments, but extracting fine-grained geographic information from such data remains challenging. In particular, fine-grained regional geolocalization is challenging because nearby areas often share coarse geographic cues. We study regional geolocalization within a metropolitan area and ask whether pretrained CLIP features are sufficient for regional discrimination, and what visual information supports performance after adaptation. Using 9,085 street-view images from eight Greater Los Angeles regions, we compare zero-shot CLIP, frozen-encoder readouts, partial encoder updating, Low-Rank Adaptation (LoRA), and full fine-tuning. Frozen readouts remain near the 39.03% zero-shot accuracy, whereas encoder adaptation achieves 75.94-82.10%. Full fine-tuning also reduces the mean distance to the predicted region center from 12.30 km to 3.86 km. We probe these gains through semantic cue removal, appearance reduction using edge maps and blur, and scene-configuration disruption using patch scrambling. Adapted models achieve higher edge and blur accuracy and switch 42.92-45.56% of predictions after scrambling, compared with 10.79-14.60% for frozen methods. However, adaptation does not improve the fraction of performance retained after appearance reduction, while vegetation and sky remain influential. A Caltech101 control further shows that scrambling sensitivity is not unique to geolocalization. Overall, encoder adaptation substantially improves nearby-region discrimination and is associated with greater sensitivity to intact scene configuration, without evidence that coarse structure alone becomes sufficient for prediction. These conclusions concern viewpoint variation near known locations rather than geographically disjoint generalization.

0.29

MIND the Gap: A Geographic Implicit Neural Representation with Adjustable Spatial Scale

cs.CV cs.LG ·2026-09-21 ·Isaac Corley, Arjun Rao, Esther Rolf, Konstantin Klemmer, Evan Shelhamer +5 more
similarity 0.64
Abstract · https://research.taylorgeospatial.org/mind

Geographic measurements are often sparse, leaving large areas without labels for the quantities we want to map. Geographic implicit neural representations (INRs) provide coordinate-based embeddings that can be combined with sparse labels to predict at unsampled locations without satellite imagery at inference. Yet existing INRs are largely evaluated with random holdouts, leaving their ability to generalize across larger geographic gaps unclear. We introduce Matryoshka Implicit Neural Distillation (MIND), a geographic INR whose spatial granularity can be adjusted after training. MIND distills several pretrained geospatial models using nested supervision at increasing embedding dimensions, dividing the representation into contiguous chunks. Early chunks capture broad spatial patterns, while later chunks add increasingly local variation. Downstream models can retain only the leading chunks or use our Chunked Penalty to reduce reliance on later chunks without retraining the INR. We also introduce CoordBench, comprising $52$ datasets and $78$ targets with both random and regional holdouts at multiple spatial scales. Across CoordBench, fine-scale features help most when labels are nearby, while smoother representations generalize better across larger geographic gaps. MIND with the Chunked Penalty achieves the highest aggregate regression and classification performance among tested INRs and the highest overall performance under regional holdout. These results show that geographic representations should be evaluated and adapted according to the spatial separation between labeled and prediction locations.

0.55

DAC-MACE: Context-Conditioned Expert Routing for Compact Scene Coordinate Regression in Camera Relocalization

cs.CV ·2026-11-06 ·Dingying Fan, Mingkai Liu, Haohua Que, Haojia Gao, Xianliang Huang +3 more
similarity 0.64
Abstract · 10 pages, 6 figures. Submitted to IEEE Sensors Journal

A camera becomes an absolute pose sensor when its images can be relocalized against a prebuilt map of the scene. On mobile robots and augmented-reality devices, that map must be small and quick to build. Scene coordinate regression (SCR) stores the scene in a compact network that predicts a 3D coordinate for every pixel, and mixture-of-experts (MoE) variants extend its capacity to larger outdoor scenes. In existing MoE-based SCR, however, the router assigns each pixel to an expert from the same local descriptor that the experts regress from, so repeated facades and windows at different locations receive similar routing inputs. We present DAC-MACE, which conditions soft expert routing on recalibrated and contextualized features. A residual context adapter (RCA) reweights the channels and locations of frozen encoder features, a global context encoder (GCE) mixes them with multi-scale self-attention, and an image-level pooled descriptor is added to the router input. A dual-threshold RANSAC--Levenberg--Marquardt estimator recovers the pose. On Cambridge Landmarks, DAC-MACE attains a mean per-scene median translation error of 13.0 cm over three seeds, the lowest among the RGB-only SCR methods compared (GLACE 13.6 cm, MACE 14.4 cm), and reduces the error on the largest scene from 24 to 19 cm relative to MACE. In a cumulative study, the MoE head, GCE, RCA, and the dual-threshold estimator lower the mean error from 17.0 cm to 14.4, 14.0, 13.6, and 13.0 cm. On 7-Scenes, adding GCE to a single ACE head raises Recall@(5 cm, 5$^\circ$) from 86.56% to 87.60%.

0.30

Selective Positional Allocation for Noise-Robust Query Retrieval in Small-Object Detection

cs.CV ·2026-10-06 ·Yangchen Zeng
similarity 0.63
Abstract · 12 pages, 4 figures, 4 tables

In aerial and remote-sensing imagery, detecting tiny targets is frequently hampered less by the representational capacity of the backbone network than by the manner in which positional priors are distributed across ill-defined foreground and background regions. We introduce SPAR (Selective Positional Allocation and Retrieval), a detection framework that re-conceptualizes positional cues as an allocatable resource: they are intensified at semantically meaningful locations while responses driven by scene clutter are actively attenuated. At the heart of SPAR lies Selective Positional Embedding (SPE), an interpretable embedding mechanism paired with an activation-bar visualization that facilitates direct inspection and hyper-parameter tuning. The encoder branch, termed MOHFE, fuses heatmap-modulated positional signals with category-specific and box-level evidence to yield more compact multi-scale features; the decoder branch, termed HIDQ, employs a foreground gating operation to discard background-dominated candidates prior to query decoding. To cope with the extreme sparsity of small-object evidence, we additionally design DPSConv (Dual-Path Sampling Convolution), which augments retrieval-oriented representations through two geometrically complementary sampling routes. Notably, all heatmap supervision is derived from existing detection objectives and consumed exclusively during training, requiring no auxiliary heatmap annotations and leaving inference as an ordinary forward pass with no additional gradient overhead. Comprehensive evaluations on DOTA, DIOR, VisDrone, PASCAL VOC, NWPU VHR-10, and AITOD demonstrate the effectiveness of the proposed design: relative to the baseline, SPAR attains an improvement of up to 9.1 percentage points, shortens the decoder from eight layers to three, and reduces model parameters by 59.4% (66.3M versus 163M).

0.26

PlacePulse: Learning Transferable Place Representations from Structural Geographic Data

cs.LG ·2026-10-09 ·Lucía García-Duarte, Miguel Álvarez, Diego Vicente, Javier Pérez Trufero, Javier de la Torre
benchmark similarity 0.63
Abstract · 42 pages, 11 figures, 12 tables

Geospatial machine learning often depends on task-specific feature engineering over large collections of sociodemographic, business, and environmental variables. Geospatial foundation models offer reusable representations of place, but leading approaches typically encode dynamic signals, such as mobility and search activity, or visual signals from satellite imagery. Few models represent the static structural composition of places (who lives and works there, which businesses operate there, and which risks they face) as transferable embeddings at fine spatial resolution. We introduce PlacePulse, a structural geospatial foundation model that compresses 572 attributes, expanded to 2,288 inputs through H3 neighborhood aggregates, into 256-dimensional unit-norm embeddings for 1,055,537 H3 resolution-7 cells in the United States. We also introduce a reusable two-axis benchmark that separates intrinsic representation quality from downstream predictive utility across multiple targets and spatial supports. Intrinsically, the embeddings reconstruct standardized inputs with mean $R^2 = 0.853$, use 251 of 256 approximately decorrelated dimensions, and show strong spatial organization, with Moran's $I = 0.93$ on the leading principal component and autocorrelation persisting beyond the input radius. Extrinsically, across 74 held-out targets from 12 external sources and four spatial supports, PlacePulse outperforms a dimensionality-matched feature-selection baseline on targets unrelated to the training inputs and is competitive with Google's Population Dynamics Foundation Model, leading on health, economic, election, economic-mobility and social-vulnerability outcomes, with relative strengths that vary by domain. These findings position structural composition as a distinct and complementary modality for geospatial foundation models, not simply a fallback where behavioral or imagery data are unavailable.

0.23

Towards Consistent Geospatial Representations: Aligning One Encoder to Many Foundation Models

cs.CV ·2026-10-20 ·Arthur Ouaknine, David Rolnick
publishedbenchmark similarity 0.63
Abstract · Accepted at NeurIPS REO Workshop 2026

Geospatial foundation models (GFMs) differ widely in pretraining objective, modality and resolution, and none leads across benchmarks. We ask whether their representations are complementary, and whether a single encoder can inherit that complementarity. We introduce Pangaea-Latents, an extension of the Pangaea benchmark providing frozen embeddings of $5$ GFMs over its $11$ datasets, along with reproduced baselines, and find GFM embeddings to be neither redundant nor disjoint. We then propose GeoMixture, a preliminary framework to align a ViT to several GFMs per dataset through a multi-representation alignment (mREPA) objective. Our encoder ties the best GFM in average rank across nine Pangaea datasets, trading peak performance for cross-task consistency.

0.26

Text-Assisted Event Stream Visual Place Recognition via Multi-Scale Spatiotemporal Optimal Transport

cs.CV ·2026-11-01 ·Lan Chen, Xingxing Xiong, Xiao Wang, Ziyan Zhang, Zikang Yan +3 more
similarity 0.63
Abstract · 17 pages including appendix, 4 figures. Submitted to ICLR 2027

Benefiting from the high dynamic range, high temporal resolution, and low energy consumption of event cameras, low-altitude economic research leveraging event cameras has become a popular topic. Several existing attempts introduce natural-language descriptions into visual place recognition (VPR) to mitigate the sparsity of structural spatial information inherent to event cameras. Nevertheless, these methods still fail to fully address two core challenges: effectively encoding event streams and semantically aligning event streams with text. In this paper, we propose a multi-scale spatial-temporal optimal transport approach, MSTOT-EPR, to address these challenges. Specifically, given event streams and scene text information, we first adopt a visual encoder and a text encoder to extract respective feature representations. We then apply optimal-transport constraints to achieve cross-modal alignment. We then design a cross-modal bottleneck fusion module to further fuse bimodal features. Furthermore, we develop multi-scale spatial optimal transport and inter-frame temporal optimal transport modules to mine high-quality spatial-temporal event features for scene recognition. Extensive experiments on the NYC-Event-VPR and EPRBench datasets fully validate the effectiveness of our proposed framework for event-based VPR. The source code of this paper will be released upon acceptance.

0.25