CS Paper Radar

MEOX: Modality-Conditioned Routing Before Fusion in a Compact Earth Observation Encoder

Mohanad Albughdadi

cs.CV ·submitted 2026-09-04 16:59 UTC · v3

Abstract

Multimodal Earth Observation models often fuse radar and optical inputs early, limiting sensor-conditioned processing, or maintain separate sensor streams through the encoder, increasing computational cost. We present MEOX (Multimodal Earth Observation with eXperts), which delays fusion until after a shared mixture-of-experts block that processes Sentinel-2 and ascending and descending Sentinel-1 streams independently. Co-located patches are then fused with validity-aware weights into a single sequence processed by fourteen shared blocks, reducing the sequence length from 3N + 5 to N + 5 tokens for N patches. We pretrain the 2.939M-parameter encoder on 1.228M MMEarth64 samples using masked radar-optical reconstruction. Under the reported frozen-transfer protocols, MEOX reaches 90.56% average class accuracy on EuroSAT and 64.42% mean intersection over union (mIoU) on cashew segmentation, 2.26 and 5.02 points above the strongest reported cross-sensor mixture-of-experts (CSMoE) variants. Full-encoder adaptation increases BigEarthNet micro-average precision from 57.22% to 72.95%. In a controlled same-corpus test using synthetic random patch removal, masking 75% of Sentinel-2 patches reduces WorldCover mIoU from 30.45% to 26.53% when both Sentinel-1 streams remain available; with Sentinel-2 alone, mIoU falls to 13.81%. For identical zeroed optical patches, declaring them missing through the validity mechanism improves mIoU from 19.42% to 26.53%. These results characterize a compact route-then-fuse encoder that retains sensor-conditioned processing and uses complementary radar observations when optical inputs are incomplete.

Closest papers

Nearest neighbours by meaning across the whole index.

Parameter-Efficient Adaptation of Self-supervised Vision Backbones Survives Distribution Shift in Radar Micro-Doppler HAR

cs.LG ·2026-11-19 ·Keryan Chelouche, Petr Dobias, Nistor Grozavu, Olivier Romain
code/data similarity 0.59
Abstract · Submitted to ICASSP2027

Micro-Doppler radar enables contactless, privacy-preserving human activity recognition (HAR), but deployment robustness remains underexplored. We compared full fine-tuning, frozen probing, and parameter-efficient fine-tuning (PEFT) using subject-disjoint in-distribution cross-validation and few-shot cross-age, cross-orientation, and cross-dataset transfer. Full fine-tuning suffered negative cross-dataset transfer, while frozen features under-adapted on cross-orientation. Only PEFT avoided both failures. We paired DINOv3 with all-layer principal-singular-direction adaptation (PiSSA), to our knowledge the first application of a self-supervised image foundation backbone to radar micro-Doppler HAR. DINOv3-PiSSA ranked first among PEFT methods in every evaluated setting. Under our protocol, its Small variant matched the in-distribution accuracy of the prior Base-size PEFT state of the art using 4.5x fewer total parameters, with only 3% trainable. It surpassed a size-matched variant of that baseline in every evaluated setting, with transfer gains increasing with shift severity: 2.1, 6.6, and 8.9 percentage points, respectively. Ablations attributed improvements to both pretraining and adapter recipe, with the pretraining contributing most under compound shift. Code and protocol are available at https://github.com/KeryanChelouche/udoppler-ssl.

0.32

Warping Earth Observations for better ice labeling in the Marginal Marginal Ice Zone

cs.CV ·2026-08-12 ·Tom Kelly, Martin S. J. Rogers
publishedbenchmarkdataset similarity 0.59
Abstract · ECCV Workshop paper; BEAM 2

Multimodal satellite imagery provides complementary information for Earth Observation, but accurately combining heterogeneous sensors remains challenging in dynamic environments. Fast-changing regions, such as the Antarctic marginal ice zone, cannot fully exploit multimodal information from different satellite sensors because surface features move between image acquisitions. This spatial and temporal mismatch challenges effective perceptual grounding, violating the assumption of pixel-level correspondence that underpins most multimodal reasoning and downstream classification pipelines. Antarctic sea ice provides a challenging benchmark due to the rapid, heterogeneous drift of individual ice floes and the differing responses of sea ice to radar, visible and thermal sensing modalities. Accurate, dense supervision of sea ice remains scarce because generating pixel-wise labels requires time-consuming expert interpretation of noisy data, leading to historical reliance on coarse-resolution maritime ice charts for model training. This paper presents a novel architecture based on mutual information warping to align multi-satellite (Sentinel-1 and MODIS platforms) multimodal (visible, thermal, radar) satellite scenes. To demonstrate the approach, we introduce a sparse expert-labeled dataset of 2,088 pixel-wise annotations (7,046 expert point classifications) located at the ice-water margin interface across 43 scenes. Our results demonstrate that spatially grounding and aligning modalities prior to segmentation improves classification accuracy, and enables accurate, dense sea ice segmentation from sparse point-wise supervision.

0.19

Learning Multimodal Sensor Fusion with JEPA Representations for Breathing Activity and Pose Recognition

cs.CV cs.AI ·2026-10-11 ·Yunxiang Luo, Constantino Álvarez Casado, Xiaoting Wu, Nhi Nguyen, Le Nguyen +2 more
hot direction similarity 0.59
Abstract · 6 pages, 2 figures, 3 equations, 4 tables, conference paper

Predictive video representations do not by themselves specify how a multimodal system should handle missing sensors. We transfer frozen V-JEPA2 features to a set of heterogeneous sensors comprised by RGB video, depth video, and radar time series. A shared fusion model uses modality queries, fixed sensor slots, and explicit availability to classify breathing activity and pose from all seven non-empty sensor subsets. The contribution is a controlled evaluation of subset supervision, pooling capacity, and representation transfer on 39 OMuSense-23 subjects with five subject-group folds and three seeds. With matched downstream parameter counts, the final model reaches recording-level activity macro-F1 of $75.23%\pm0.60$ averaged across seven subsets. Supervising all subsets in each batch improves the matched query-pooling control by 1.923 percentage points (conditional 95% subject-bootstrap interval [0.647, 3.209]), while query pooling shows no clear advantage over matched mean pooling under balanced supervision. Radar probes support transfer beyond pooled wavelet pixels, 64-token inference approximately preserves mean activity performance, and the tested Consensus-JEPA loss adds no measured benefit against its exact control.

0.36

Quantized Edge Inference of a Self-Supervised SAR-Optical Encoder on Commodity x86 Hardware

eess.IV cs.CVcs.LG ·2026-10-27 ·Christian Sanchez-Leon, Dongming Chen
similarity 0.59
Abstract · 3 pages, 1 figure, 3 tables

Self-supervised vision transformers are increasingly used as reusable encoders for Earth observation, but their inference cost on the hardware available in the field is rarely characterized. We deploy a pretrained SAR-optical masked-autoencoder encoder on three commodity x86 nodes, from a low-power CPU without AVX2 to an AVX2+VNNI reference, at FP32, FP16 and INT8 on the CPU and, where present, the integrated GPU, and report latency, energy (RAPL and wall plug) and linear-probe accuracy. Quantization is accuracy-neutral: INT8 changes micro-mAP by only +0.0006, within the seed spread, so precision becomes a pure cost decision. INT8 lowers CPU latency on every node, by 1.45x even on the CPU without AVX2, and CPU energy wherever it is measured, and the VNNI reference (1.53x) suggests a further gain beyond AVX2. The Pareto-optimal device shifts from the integrated GPU, which is 4.38x faster than the CPU on the weakest node, to the CPU as the CPU class improves.

0.24

SAR-TopoGuard: Training-Free, Speckle-Aware Topological Verification for Structure-Preserving SAR-to-Optical Translation

cs.CV cs.AI ·2026-11-14 ·Satabarto Sarkar, Debojyoti Ghosh, Subhranil Biswas, Kaustav Mandal, D. Sobya
publishedcode/data similarity 0.58
Abstract · 10 pages, 4 figures, 7 tables. Accepted manuscript of the article published in IEEE Geoscience and Remote Sensing Letters (2026). \c{opyrigh…

SAR-to-optical translators often synthesize detail that the radar measurement does not support, and pixel metrics alone cannot reveal it. We propose SAR-TopoGuard, a training-free inference-time layer for frozen generators: a stochastic sampler draws K candidates (Monte Carlo dropout for GANs, seed variance for diffusion), a speckle-aware verifier ranks them by gradient-magnitude correlation with the SAR input and a hallucination-frequency index, and a SAR-guided frequency gate admits high-frequency detail only where SAR edges support it. With all five hyperparameters frozen and three training seeds, we evaluate on 500 scene-disjoint SEN1-2 test patches, 500 patches of a held-out summer scene, and 200 cross-dataset SEN12MS winter pairs. The layer improves Pix2Pix by +0.060, +0.012, and +0.077 dB PSNR and by +0.0034, +0.0057, and +0.0135 SSIM, respectively (Wilcoxon p < 1e-6), and it improves CycleGAN and CUT on all three sets (+0.013 to +0.076 dB). It raises edge precision at the cost of edge recall (edge F1 -1.3% on the test set). On the latent diffusion model COP-GEN, it adds +0.33 dB (base) and +0.16 dB (Beta) on the winter set, the only set with dual-polarization SAR. An ablation shows that the gate produces the gain for MC-dropout GANs, whose candidates are nearly identical, whereas verifier selection contributes up to +0.26 dB for diffusion, whose samples differ. The layer has zero trainable parameters and adds 0.4 ms per patch beyond the K=5 generator passes.

0.42

TOP: Transmission-Oriented Remote Sensing Pansharpening

eess.IV cs.CV ·2026-11-04 ·Zi-Han Cao, Jing-Ze Liang, Shao-Jie Wu, Jin-Cheng Di, Liang-Jian Deng +1 more
code/dataunusual similarity 0.57
Abstract

Pansharpening has been extensively studied as a fundamental image fusion task in remote sensing. However, most existing methods focus on improving fusion quality under the implicit assumption that high-resolution observations have already been fully downlinked to the ground. As a result, existing models are highly vulnerable to perturbations introduced by lossy compression used for downlink transmission. To address this limitation, we present, to the best of our knowledge, the first transmission oriented pansharpening (TOP) framework, which unifies compact latent coding, onboard pansharpening, and efficient downlink transmission within a single satellite-toground pipeline. TOP encodes remote-sensing observations into a compact binary latent space, where a lightweight onboard fusion module can efficiently predict the fused binary latent code. Since only the compact fused code is downlinked, the transmission payload is substantially reduced. Overall, TOP enables low-payload downlink and lightweight onboard processing while maintaining high fusion quality under severe bandwidth constraints. Experiments across multispectral and hyperspectral settings show that TOP consistently achieves superior fusion quality at higher compression ratios than representative pansharpening methods paired with standard lossy codecs. For example, on GaoFen-2, TOP attains a fusion PSNR of 31.4 dB at a downlink compression ratio of 50x. This design also points toward transmissionoriented processing for other remote-sensing tasks. The code will be made available at https://github.com/JingZe-Liang/TOP.git.

0.43

SyntheIce: A Synthetic Radargram Annotation Benchmark for Englacial Layer Reconstruction

cs.CV ·2026-11-10 ·Muhammad Behroze Hassan, Bayu Adhi Tama, Sanjay Purushotham, Vandana P. Janeja
publishedbenchmark similarity 0.57
Abstract · 10 pages, 10 figures, 3 Tables, accepted in IEEE International Conference on Big Data (BigData) 2026

Englacial layers in radio-echo-sounding radargrams (subsurface ice images) encode the climate and flow history of ice sheets and glaciers. However, large-scale extraction is hindered by time-consuming manual annotation and semi-automated detectors that produce fragmented rather than continuous horizons. We address two challenges: reconstructing complete layers from partial detections across radargram profiles and validating layer continuity across adjacent tile boundaries. We introduce a slope-aware greedy fragment-linking algorithm that reconstructs full-width layers using row-similarity gating and an exponentially weighted moving average (EWMA) slope predictor. Each reconstruction receives a content-based hash ID for stable cross-radargram identification. For quantitative evaluation, we generate SyntheIce, a synthetic dataset of 600 radargrams spanning multiple complexity levels and realistic noise conditions, including 150 wide radargrams divided into four adjacent tiles. All samples include exact ground-truth annotations. We evaluate the dataset using XCheck, a consistency-based validation framework, extending its original two-image setting to four consecutive tiles and enforcing continuity at every boundary. This demonstrates the dataset's utility for developing and evaluating real-world annotation algorithms. Our findings extend beyond ice science: when spatial features are fragmented but retain consistent local geometry, trajectory-based linking is more reliable than appearance-based matching. Evaluation against expert-labeled real radargrams demonstrates high precision, recall, F1-score, and mean IoU, supporting the method's transfer to real-world data.

0.30

Shift Aware Transfer Learning with Adaptive Dual-Encoder Fusion for PM Forecasting in Data-Limited Environments

cs.AI ·2026-08-14 ·Shahab Band, Hamed Mohammadi
similarity 0.57
Abstract

Short-horizon forecasting of fine particulate matter (PM2.5) remains difficult when observations from the target domain are limited and the statistical properties of the source and target domains differ. In these settings, models trained only on local data may not capture complex temporal dynamics, while direct transfer learning can result in negative transfer. This study develops a shift-aware dual-encoder transfer framework that combines source-domain knowledge with target-specific representation learning. The source encoder was pretrained using hourly observations from 10 U.S. monitoring locations. The framework was then adapted and evaluated using two years of hourly observations from 77 stations in Taiwan under a chronological train-validation-test protocol. Among the four principal baselines, the frozen-source dual-encoder model achieved the best performance, with MSE = 21.8960, MAE = 3.1597, and R^2 = 0.8725. This corresponds to an MSE reduction of approximately 7.1% relative to TL-v1 and 4.1% relative to TL-v2. The ablation analysis showed that removing the Taiwan-specific branch caused the largest decline in performance. Allowing the source encoder to adapt produced the best overall result, with MSE = 21.6575, MAE = 3.1383, and R^2 = 0.8739. SHAP analysis indicated that predictions were driven mainly by recent PM2.5 observations and meteorological variables related to pollutant transport and dispersion. These results suggest that source-domain knowledge is most effective when target-specific information is preserved and the transferred representation is allowed to adapt under target supervision.

0.04