CS Paper Radar

When to Adapt: Multi-Signal Domain Shift Detection for Efficient Training-Free Adaptation in Open-Vocabulary Segmentation

Michele Antonazzi, Alejandra C. Hernandez, José Araujo, Olov Andersson, Patric Jensfelt

cs.RO cs.CV ·submitted 2026-09-29 13:53 UTC · v1
benchmark

Abstract

Robust and reliable perception is essential for autonomous robots operating in real-world environments, particularly in long-term missions where environmental conditions may change significantly over time. Although recent advances in Visual Foundation Models (VFMs) have improved open-vocabulary semantic segmentation, these models can still suffer from domain shift, which can significantly degrade performance if they are not adapted to the current environment. Training-free domain adaptation is a relevant paradigm for adaptation, consisting of adjusting the model online using lightweight adapters. Recent approaches apply this on a per-frame basis, which is impractical for deployments on resource-constrained robotic hardware. To tackle this, we propose a multi-signal domain shift detection method for training-free continual test-time adaptation (TF-CTTA) in open-vocabulary segmentation. Our method leverages temporal coherence across consecutive frames by monitoring and combining complementary aspects of domain shift (visual change, adapter mismatch, and semantic drift) to trigger adaptation only when needed. We validate our approach on a benchmark including indoor and outdoor environments and using real robotic data. We demonstrate that our approach maintains segmentation accuracy while substantially reducing adaptations, making training-free adaptation practical and feasible for long-term, real-world robotic deployments.

Closest papers

Nearest neighbours by meaning across the whole index.

Towards Continual Test-Time Adaptation of Vision-Language Models in Open-Vocabulary Semantic Segmentation

cs.CV cs.LG ·2026-08-30 ·Chandler Timm C. Doloriel, Yunbei Zhang, Sarthak Kumar Maharana, Muhammad Salman Siddiqui, Tor Kristian Stevik +3 more
code/data similarity 0.70
Abstract · under review. code available at https://github.com/chandlerbing65nm/DAF.git

Open-vocabulary semantic segmentation (OVSS) relies on vision-language alignment to recognize arbitrary text-defined categories, yet this alignment is fragile under continual test-time distribution shift. Our diagnostic analysis reveals that entropy minimization drives patch-level class collapse, continual updates erode vision-language alignment, and redundant gradients from low-shift samples waste computation. We propose Diversify, Anchor, and Filter (DAF), a stabilization framework that augments entropy-based adaptation with a marginal diversity loss that resists collapse, a cross-modal anchor consistency loss that constrains feature drift relative to a frozen source model, and feature salience filtering that skips low-value backward passes to offset part of the source-anchor overhead. We evaluate on five datasets spanning natural scenes, autonomous driving, underwater imagery, and remote sensing with their corrupted variants. Across the evaluated continual shifts, DAF remains stable where entropy minimization collapses, improving mIoU by over 8 points on Pascal VOC20-C, over 9 points on LoveDA, and over 3 points on Foggy Cityscapes compared to the source model, and is robust to aggressive adaptation and learning rate choices.

0.53

PISA: A Pseudo-Individual Source-Domain Feature Adaptation Framework for Test-Time Open-Vocabulary Object Detection

cs.CV ·2026-08-14 ·Ziyan He, Xiongtai Yang, Tao Wang
similarity 0.69
Abstract

Open-vocabulary object detection test-time adaptation (OVOD-TTA) aims to address the performance degradation that pre-trained base models suffer when encountering image-domain shifts. Existing source-free OVOD-TTA methods rely either on refined test-time information for re-scoring or on pseudo-labels for self-training, leading to significant accuracy degradation when initial predictions are poor. Meanwhile, most conventional source-domain estimation methods recover abstract, sparse representations suitable for the classification task, but fail to capture the dense, concrete features required for detection. To address these issues, we propose PISA, a novel source-free OVOD-TTA method that can be seamlessly integrated into open-vocabulary visual backbones. The core components of our method are the Corruption-Invariant Feature Extractor (CIFE), the Feature Alignment Module (FAM), and a multi-scale alignment framework (BAA). To capture detection-suitable features, we develop CIFE to exploit the invariance of CLIP's visual features across corrupted images, ensuring robustness against various corruptions. We further develop FAM and BAA for the pre-training and adaptation to transform the corruption-invariant features into pseudo-individual source-domain features that are close to the original source-domain features. In this way, dense and concrete pseudo-individual source-domain features are used for supervision instead of unreliable pseudo-label signals. Experiments on the corrupted VOC-C, COCO-C, and LVIS-C benchmarks across three base models demonstrate that PISA substantially improves both the localization precision and the category recognition accuracy of the original models. Notably, PISA achieves state-of-the-art performance without requiring access to source-domain data, surpassing existing methods by 3.92% in AP@50% on COCO-C.

0.04

Self-Adaptive VLA for Robust Robot Deployment

cs.RO cs.CV ·2026-09-24 ·Hongxin Zhang, Chunru Lin, Tsun-Hsuan Wang, Zhenjia Xu, Chuang Gan
similarity 0.68
Abstract

While Vision-Language-Action (VLA) models demonstrate impressive capabilities in robotic manipulation, their memoryless nature renders them brittle to test-time environment shifts, particularly hardware shifts caused by wear or imperfect calibration. Enabling these models to self-adapt during deployment without requiring continuous on-site recalibration remains a critical bottleneck for real-world scalability. In this work, we introduce Self-Adaptive VLA, a novel post-training recipe that enables the policy to iteratively adapt to deployment-time hardware shifts leveraging its own rollouts as context. To do so, we first collect policy rollouts under deliberately injected hardware shifts. We then transform the base policy's training data into shift-conditioned expert demonstrations by pre-compensating the expert actions for these known shifts. Next, we introduce a lightweight, plug-in context encoder that compresses the context, including visual observation, proprioception, and actions in the shifted environment, into a latent context token. This token modulates the policy through adaptive layer normalization (AdaLN). Furthermore, we find that context tokens can be ensembled, allowing the policy to iteratively self-correct and mitigate failures step by step. Extensive experiments across four precision-critical bi-manual and dexterous manipulation tasks show that Self-Adaptive VLA recovers over 80% of the base policy's performance under hardware shifts, such as actuation bias and joint encoder offsets. Moreover, Self-Adaptive VLA enables more robust deployment to new workstations compared to the base policy. Our approach provides a pathway for robust large-scale real-world robot deployments and easier maintenance. See videos at https://icefoxzhx.github.io/self-adaptive-vla.

0.45

Understanding Temporal Semantic Stability in Open-Vocabulary UAV Perception through Metric 3D Fusion

cs.CV cs.RO ·2026-08-23 ·Saurbh Singh Jamwal
published similarity 0.68
Abstract · Accepted at the 17th Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP 2026)

Recent open-vocabulary segmentation models have significantly advanced semantic perception for UAVs, for aerial and remote-sensing imagery, yet predictions from moving platforms can remain temporally inconsistent across repeated observations of the same physical scene. While semantic segmentation is typically evaluated on single images, temporal reliability remains less explored despite its importance for long-horizon robotic perception. In this work, we investigate temporal semantic stability by associating frame-wise predictions with persistent world-space locations through metric 3D fusion. We introduce a voxel-level evaluation framework that jointly characterises final semantic agreement, Semantic Belief Drift (SBD), Observation Persistence (OP), and semantic uncertainty from accumulated evidence. Experiments on UAVid-3D-Scenes reveal substantial frame-wise semantic variation and show that high aggregate world-space agreement can overstate temporal stability when many locations have limited repeated-observation support. Persistence-stratified analysis shows that recurrent voxels expose substantially greater semantic disagreement, while higher-persistence voxels exhibit lower direct consecutive semantic-evidence divergence. This behaviour is observed across two segmentation backbones and remains consistent under variations in voxel resolution, geometric association, and temporal sampling density. Overall, conditions that reduce world-space recurrence can increase apparent aggregate stability, demonstrating that semantic consistency must be interpreted together with observation support. Our findings establish observation persistence as an essential conditioning variable for evaluating long-horizon semantic reliability and provide a framework for analysing temporal semantic behaviour beyond image-level prediction.

0.23

Pro-Bench: Prompt-Robust Open-Vocabulary Visual Grounding Across Real-World Heterogeneous Environments

cs.CV cs.RO ·2026-09-22 ·Linus Nwankwo, Muslim Alaran, Christian Rauch, Stanley Chukwuebuka Obilikpa, Elmar Rueckert
benchmark similarity 0.67
Abstract

Open-vocabulary visual grounding enables robots to localise task-relevant entities from natural-language queries without dependence on predefined perceptual taxonomies. However, existing benchmarks largely rely on short category labels and web-scraped imagery, leaving it unclear whether open-vocabulary models can robustly ground diverse queries and visual conditions under real deployments. We introduce \textbf{Pro-Bench}, a prompt-conditioned benchmark for open-vocabulary visual grounding in heterogeneous, real-world environments. Pro-Bench includes $13k+$ RGB frames from independent robotic domains (subterranean, industrial, indoor, outdoor, urban), with $74.5k$ manual instance annotations and $515$ target queries covering categorical, attributive, relational, affordance, state, part-whole, negative, and compositional semantics. We benchmarked $16$ open-vocabulary model configurations in strict zero-shot inference, measuring localisation accuracy across IoU thresholds, end-to-end inference latency, prompt-induced performance variation, and target recovery consistency. Our results show that prompt-robustness is strongly architecture-dependent. Most model configurations ($10/16$) perform best with short category labels, whereas free-form queries yield the highest accuracy for only one. Moreover, similar aggregate mAP can conceal substantial differences in consistent target recovery across reformulations. Pro-Bench enables systematic evaluation of these gaps and supports prompt-robust visual grounding. Pro-Bench: https://pro-bench.github.io/.

0.25

Test-Time Selective Adaptation for Cross-Modal UAV Object Tracking with Token Shift

cs.CV ·2026-11-11 ·Yun Xiao, Hailong Ye, Zhicheng Zhao, Lei Liu, Amir Hussain
similarity 0.66
Abstract

Cross-modal unmanned aerial vehicle (UAV) object tracking faces modality-induced appearance discrepancies and abrupt target-position shifts that can arise from modality switching, UAV motion, and inter-sensor offsets. Although existing methods model modality and spatial variations, they primarily rely on offline representation learning and offer limited localization-oriented adaptation during testing. To address these challenges, we propose Test-Time Selective Adaptation with Token Shift (TSAT), which derives spatial and temporal adaptation signals from unlabeled test sequences for online optimization. Specifically, we design a controllable token shift auxiliary task, which imposes explicit spatial correspondence between predefined token translations and resultant localization offsets, generating stable annotation-free spatial adaptation signals. Furthermore, we introduce a temporal backward prediction auxiliary task to exploit frame-wise temporal dependencies, effectively mitigating cumulative localization errors in continuous tracking. To suppress interference from unreliable tracking samples, we develop a reliability-aware update gating strategy to screen valid adaptation signals and stabilize online parameter updates. Experiments on CM-UOT demonstrate that TSAT achieves state-of-the-art performance against existing test-time adaptation methods, exhibiting superior robustness under frequent modality switching and severe spatial displacement. Experiments across tracking frameworks and datasets further validate the effectiveness and generalization capability of TSAT.

0.26

Argos: Adapt Rich Geometric Priors for Generalizable Online Scene-Change-Detection

cs.CV ·2026-10-07 ·Ruihan Xu, Jiae Yoon, Kaichen Zhou, Ue-Hwan Kim, Luca Carlone
benchmark similarity 0.65
Abstract · More details on the project website: https://www.multyxu.com/argos/

Robots operating in dynamic environments require reliable detection of how their surroundings change over time. Existing learning-based methods largely rely on pairwise 2D image features, which struggle under large viewpoint changes and occlusions, are sensitive to noise, and show limited generalization across domains, while explicit 3D approaches typically require costly offline optimization. We show that the implicit 3D knowledge of Geometric Foundation Models (GFMs) provides a strong basis for addressing these limitations. We introduce Argos, which adapts GFM features for joint scene change detection and 3D reconstruction. To address data scarcity and take a step toward a foundation model for scene change detection, we introduce a large-scale benchmark comprising two synthetic datasets and one real-world dataset, and train jointly across diverse datasets to improve cross-domain generalization. We further introduce Argos-SLAM, a real-time system designed for robotics, which performs online change detection and change-aware 4D mapping. Across benchmarks, our framework substantially outperforms existing baselines, with gains of up to 42.01% in change IoU and 27.91% in F1, while supporting scalable deployment in changing real-world environments.

0.17

RA-VLA: Retrieval-Augmented VLA for Test-Time Adaptation

cs.RO ·2026-08-26 ·Sanghwan Jang, Minjin Jeon, Minsoo Kim, Seongjin Choi, Dongha Kim +1 more
publishedbenchmark similarity 0.65
Abstract · ICML 2026. Contact: s$.$jang@postech$.$ac$.$kr

Vision-Language-Action (VLA) models provide a versatile foundation for general robotic manipulation, yet they exhibit significant brittleness when confronted with novel task distributions. While In-Context Imitation Learning (ICIL) offers a training-free alternative, existing frameworks suffer from an adaptation bottleneck that hinders the effective translation of expert context to executable actions. This failure originates from superficial retrieval mechanisms and an inherent behavioral inertia that anchors the policy to its pre-trained priors. To address these limitations, we present RA-VLA, a retrieval-augmented VLA framework that integrates behavior-aligned context retrieval with a grounded execution pipeline. By enforcing faithful adherence to functional cues within a scalable architecture, RA-VLA facilitates seamless task adaptation while preserving inference efficiency. Our empirical evaluations across the LIBERO benchmark and a real-world UR5e environment demonstrate that RA-VLA achieves superior success rates and computational efficiency, establishing a robust framework for training-free robotic adaptation.

0.49