CS Paper Radar

CoDeRNet: Selective Cross-Task Routing under Heterogeneous Supervision for Change Detection and Captioning

Eunki Cho, Hyeon Bae Kim, Seong Tae Kim

cs.CV cs.AI ·submitted 2026-10-26 05:48 UTC · v1
top venue

Abstract

Change Detection & Captioning (CDC) aims to jointly localize changed regions and describe their semantic content from bi-temporal remote sensing images. While recent approaches have demonstrated the effectiveness of unified modeling, they primarily rely on shared representations or jointly optimized architectures, leaving task-specific communication under heterogeneous supervision less explored. However, detection and captioning differ fundamentally in their supervision granularity, leading to heterogeneous representations with uneven reliability across tasks and spatial locations. In this paper, we reformulate CDC as a selective information routing problem between heterogeneous task branches, where the key challenge is to determine when and where information should be transferred. To this end, we propose CoDeRNet, a unified framework that preserves task-specific representations while enabling bidirectional exchange of complementary information. CoDeRNet introduces a Confidence-Decomposed Routing (CoDeR) mechanism, which decomposes the routing condition into receiver-side need for support and sender-side reliability cues to guide selective transfer. Experiments on LEVIR-MCI and WHU-CDC show that CoDeRNet achieves strong overall CDC performance, particularly improving caption generation while preserving competitive change localization. Extensive analyses further show that the benefits arise not merely from joint learning or generic cross-task communication, but from explicitly modeling selective cross-task information routing. These results highlight the importance of conditional information transfer for unified CDC under heterogeneous supervision.

Comment
Accepted at NeurIPS 2026. 20 pages

Closest papers

Nearest neighbours by meaning across the whole index.

DS-BiCA-Net: Pair-Preserving Cross-Temporal Modeling for Remote Sensing Change Captioning

cs.CV ·2026-10-23 ·Licheng Zhang, Naveed Akhtar, Siew-Kei Lam
similarity 0.73
Abstract

Remote sensing image change captioning (RSICC) describes in natural language the semantic changes between two images of the same scene captured at different times. Previous methods commonly merge the pair during repeated processing or combine intra-image and cross-image relations in one operation, weakening source identity and leaving the two temporal directions implicit. Without separate modules for intra-image and cross-temporal processing and a separate path for each direction, the contribution of each part cannot be measured on its own, so it remains unclear which part matters most, and the design of later models has no such evidence to build on. To address these issues, we propose DS-BiCA-Net, which assigns the two kinds of processing to separate modules and keeps the two images as separate streams throughout the encoder. A windowed state-space module models context within each image, and a bidirectional cross-attention module compares the two images along two directed paths that share their projections. The two modules use the same pre-normalisation residual architecture and differ only in their mixers, and the two streams are fused only in the aggregation head, so each module and each direction can be removed and its contribution measured on its own. Controlled experiments on LEVIR-CC and Dubai-CC examine operator interaction, temporal direction and module order, showing that intra-image refinement carries no benefit alone and becomes beneficial only when followed by bidirectional cross-attention, an order that outperforms the reverse at matched parameter count and FLOPs. The bidirectional module also outperforms either temporal direction alone. Our method reaches an S_m^* of 82.65 on LEVIR-CC, the highest among the published results compared, and 66.72 on Dubai-CC without geometric augmentation, the second-highest among the published results compared.

0.18

From Change Captions to Change Detection: Semantic-Appearance Agreement Framework for Remote Sensing Change Detection

cs.CV ·2026-09-23 ·Yuan Qian, Jie Ma
code/data similarity 0.73
Abstract · 12 pages, 6 figures, 6 tables. Code: https://github.com/qianyuancs/SAAF

Remote sensing change detection (RSCD) is essential for monitoring land-cover changes and urban development. However, most methods demand pixel-level change masks, which are costly and time-consuming to annotate. Weakly supervised methods reduce this cost by using image-level change labels. Yet these labels indicate only whether a change occurs, leaving models to recover the location of the change and semantic meaning through additional and complex mechanisms. This missing information can be supplied directly by change captions, which describe what changes, what it becomes, and where it occurs. Therefore, we introduce change-caption-guided RSCD, using change captions as the sole task-specific supervision to learn change masks without manually annotated change masks. Our framework has two components: a caption-driven generation pipeline that produces bi-temporal remote sensing image pairs at scale with controlled changes matching each caption, and a change detector guided by the caption's transition semantics. The detector uses our Semantic-Appearance Agreement Framework (SAAF) to combine caption-grounded semantic responses with RGB differences for change localization, while text conditioning guides dense prediction. Experiments on our newly constructed Flair-RSGen dataset and WHU-CDC show that SAAF outperforms the closest reproduced limited-supervision baselines in macro-averaged IoU and F1 under the evaluated protocols. Code is publicly available at https://github.com/qianyuancs/SAAF.

0.24

HIMEC: Directional Change Representation and Fixed-Interface Decoding for Remote Sensing Image Change Captioning

cs.CV ·2026-08-12 ·Aysha Ashraf, Shaina Ashraf, Wafaa I. M. Hussin, Ali Haider, Zhi Lu +1 more
code/data similarity 0.71
Abstract · Submitted to IEEE Transactions on Geoscience and Remote Sensing (TGRS) for review

Remote sensing image change captioning (RSICC) converts bitemporal imagery into a sentence describing semantic changes. Most RSICC methods condition caption decoders directly on fused visual features, leaving intermediate change structure and decoder-interface consistency less studied. We present HIMEC, combining Directional Change Representation (DCR) with fixed-interface decoding. DCR separates signed differences into appearance-oriented, disappearance-oriented, and shared-context streams before fusion. A learned-query encoder converts the fused representation into visually conditioned change-query tokens that form the scene decoder's only sample-dependent memory. A training-only auxiliary phrase decoder supplies caption-derived supervision. With a fixed zero input, the scene decoder maintains the same interface during training and inference. Separately, we evaluate a local-to-scene cascade conditioned on teacher-forced local states during training and autoregressive states at inference. On changed LEVIR-CC validation pairs, these states have a mean cosine distance of 0.69. Regime-matched conditioning recovers most of the associated deficit, whereas permuting state correspondence causes no detectable penalty. These findings are limited to the evaluated cascade. In a matched three-seed comparison, HIMEC reaches a Consensus-based Image Description Evaluation (CIDEr) score of $142.81\pm0.60$ on LEVIR-CC, versus $139.51\pm3.40$ for direct fused-feature memory. On SECOND-CC, fixed-zero and regime-matched diagnostic conditioning reach 75.67 and 76.99 CIDEr, respectively, versus 60.77 for the mismatched cascade. The source code will be made publicly available at https://github.com/ayshaashra/HIMEC upon publication.

0.26

Selective Tool Use for Agentic Change Visual Question Answering in Remote Sensing

cs.CV ·2026-09-13 ·Yakoub Bazi, Mohamad M. Al Rahhal, Mohamed A. Mekhtiche, Mansour Zuair
code/data similarity 0.68
Abstract

Change visual question answering (Change VQA) requires understanding semantic changes across bi-temporal remote sensing images. Although vision language models (VLMs) have shown promising performance on this task, they remain unreliable when answering questions that require explicit transition statistics, area measurements, or spatial information. To address this limitation, we propose a selective tool use framework in which a single VLM either answers directly or invokes a deterministic change analysis tool to obtain question specific evidence. Specifically, the selected tool operates on bi-temporal semantic maps and returns a structured observation, which the same VLM uses to generate its final answer. To support this framework, we construct a tool augmented extension of CDVQA covering eight question families and three tools for transition, spatial, and temporal analysis. Tool use supervision and observations are derived automatically from the original semantic annotations, without additional manual labeling. We then adapt Qwen3.5-4B using Low Rank Adaptation (LoRA) to jointly learn direct answering, tool invocation, and evidence conditioned answering. Experiments on 7,164 test questions show that selective tool use with reference semantic maps improves overall accuracy from 73.77% to 88.79% and average family accuracy from 69.11% to 89.65%. When the semantic maps are predicted automatically, the framework achieves 77.47% overall accuracy and 75.06% average family accuracy. These results demonstrate the benefit of question-specific semantic evidence for Change VQA, while highlighting the influence of semantic prediction quality on the resulting performance. Code and tool-augmented annotations will be made publicly available at https://github.com/yakoubbazi/ToolChangeVQA.

0.27

From Coordinates to Candidate Regions: Temporal Change Localization via Region Selection in Remote Sensing Multimodal LLMs

cs.CV cs.CL ·2026-09-08 ·Juwan Chung, Sungjune Park, Yeongyun Kim, Yong Man Ro
top venuecode/data similarity 0.67
Abstract · Accepted to Findings of EMNLP 2026

Remote sensing multimodal large language models (RS-MLLMs) have advanced scene understanding and visual question answering over satellite imagery, yet localizing specific objects or changed regions remains challenging. Existing approaches rely on generating bounding box coordinates as token sequences, which is fragile for the small, densely packed objects common in remote sensing and increasingly error-prone when multiple targets must be localized simultaneously. In this work, we present an RS-specific formulation of the region selection paradigm, previously explored in natural-image MLLMs, and extend it to temporal change localization over multi-image sequences. Our framework employs a text-conditioned region proposal module, encodes each candidate as special tokens carrying per-frame visual features enriched with spatial and temporal cues, and lets the LLM localize targets by selecting region tokens in its response. We construct a multi-task training and evaluation suite spanning localization, referring expression, visual grounding, and understanding tasks across single-image and multi-temporal settings. Experiments show that our approach substantially outperforms coordinate-generation baselines on temporal change localization, while improving single-image visual grounding and maintaining competitive understanding performance. Oracle analysis decomposes the contributions of the region proposer and the LLM selector, providing diagnostic insight unique to this framework. Our code will be available at https://github.com/juwan-kr/RS-RegionSelect.

0.45

VPRef: A Cross-Domain Benchmark for Referring Remote Sensing Image Segmentation

cs.CV ·2026-09-15 ·Quanwei Liu, Tao Huang, Jiaqi Yang, Wei Xiang
benchmarkcode/data similarity 0.66
Abstract · 12 pages, 7 figures, 6 tables

Rapid advancements in vision-language models have propelled Referring Remote Sensing Image Segmentation (RRSIS) to the forefront of Earth observation. However, practical deployments suffer severe performance degradation under a coupled dual-drift paradigm: visual domain drift from cross-spatial-resolution mismatches and spectral variations, alongside textual logic drift from unconstrained, variable user-input granularities. To mitigate these bottlenecks, this paper establishes the first cross-domain RRSIS benchmark, designated as the Vaihingen-Potsdam Referring (VPRef) dataset, comprising 46,972 language-image-annotation triplets organized into a three-tier linguistic hierarchy. Building upon this benchmark, we develop a tailored parameter-efficient domain adaptation baseline anchored on the Segment Anything Model (SAM3) via Low-Rank Adaptation (LoRA). Our framework counteracts visual distribution discrepancies through pseudo-label-driven self-training and addresses textual logic drift via random multi-granularity text prompt mixing. Crucially, the distribution of empirical metrics across ablative variants suggests a potential decoupling between cross-modal semantic robustification and visual domain alignment, demonstrating that linguistic variance drives fine-grained semantic invariance while pseudo-label propagation governs macro-scale spatial grid alignment. Extensive benchmarks demonstrate the proposed framework achieves superior cross-domain segmentation boundaries while modifying merely 1.08\% of the foundational parameter footprint, establishing a robust baseline for future multi-modal remote sensing domain adaptation research. The dataset and code will be available at https://github.com/quanweiliu/VPRef.

0.30

QSCP: Beyond Class-Name Prompts for Query-Guided Semantic Change Parsing

cs.CV ·2026-09-27 ·Yuan Qian, Jie Ma
code/data similarity 0.66
Abstract

Traditional change detection (CD) identifies changes between bi-temporal remote sensing images, while semantic change detection (SCD) assigns predefined land-cover classes. However, mapping all changes may not meet a user's specific needs. Referring change detection (RCD) enables selective retrieval through category prompts. However, existing category-prompted RCD uses the queried category to specify the destination of a change and returns only a binary mask of the corresponding regions. Users may instead request a particular transition and paired semantic maps to understand what changed into what. Such requests require explicit source and target reasoning beyond target-class localization. To address these needs, we propose query-guided semantic change parsing (QSCP), which supports category names, synonyms, and intent-bearing sentences and returns a query-specific mask with paired temporal semantic maps. QSCP parses requests into intents and semantic slots, composes bidirectional visual evidence, and predicts both temporal states with a query-conditioned decoder. On SECOND, QSCP outperforms RCDNet on synonym, sentence, and transition queries and improves end-to-end semantic prediction over evaluated semantic baselines. WHU-CDC experiments further assess cross-dataset transfer and consistency across equivalent expressions without target-domain training. Code is available at https://github.com/qianyuancs/QSCP

0.20

Spot-the-shift: Evaluating Grounded Image Difference Captioning of Long-term Changes

cs.CV ·2026-09-09 ·Benedetta Liberatori, Nermin Samet, Paolo Rota, Matthieu Cord, Elisa Ricci +2 more
benchmark similarity 0.65
Abstract · Preprint

Long-term change understanding from images of the same place revisited over time is a challenging task with applications in map maintenance and urban infrastructure monitoring. Prior work addresses it either through pixel-level prediction or difference captioning, neither of which is sufficient to reliably measure how well models detect and describe such changes. We introduce SPOT-THE-SHIFT, a human-verified benchmark for grounded image difference captioning of long-term changes in real-world driving scenes. Our benchmark provides natural language captions and spatial masks for structural changes across each image pair. We further propose an evaluation protocol that reliably assesses models' captioning ability, validated through human studies. Benchmarking state-of-the-art MLLMs, we find that models struggle with the fine-grained multi-image spatial capability required for this task. Finally, we develop a synthetic data generation pipeline that improves an off-the-shelf MLLM without sacrificing general capabilities.

0.09