CS Paper Radar

Overlapping Visual Grouping Without Semantic Priors

Teemu Saukkio, Hashem Haghbayan, Juha Plosila

cs.CV ·submitted 2026-09-23 06:38 UTC · v1

Abstract

Most computer-vision systems organize visual input toward a predefined interpretation, such as semantic categories, prompted regions, learned object-like representations, or a single spatial partition. This work considers an earlier stage of visual organization: the formation of candidate perceptual units directly from sensor measurements before their identity, meaning, or task relevance is known. We introduce Domain Parent Grouping (DPG), a sensor-grounded grouping method in which complementary measurement relationships are represented in separate processing domains. Spatially connected groups formed within these domains are related through cross-domain overlap, yielding a non-exclusive grouping representation rather than a single mutually exclusive segmentation. This representation retains broader and more localized groups, as well as alternative grouping boundaries over the same image locations, simultaneously available. DPG also includes a native mechanism for reprocessing selected group content, in which input-relative measurement ranges allow the observational resolution to change while preserving previously formed groups. DPG is implemented using three domains representing locally contextualized luminance, direct chromatic relationships, and contextual chromatic relationships. Experiments on the BSDS500 dataset demonstrate the benefit of combining the three domains. The results further show that DPG forms measurement-supported groups corresponding to low-level image structure, and that these groups exhibit measurable correspondence with human-annotated regions and boundaries. This demonstrates that structured visual organization can emerge directly from relationships among sensor measurements.

Comment
39 pages, 13 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Geometric Similarity in VLM Low-Level Vision Representations

cs.CV cs.AI ·2026-10-01 ·Shao-Jun Xia, Huixin Zhang, Zhen Lei, Anlan Sun, Yuner Zhang +1 more
similarity 0.57
Abstract · First version: 10 pages

Vision-language models (VLMs) have emerged as powerful candidates for universal vision backbones, with representative architectures including autoregressive (AR) models and diffusion transformers (DiTs). Yet, adapting them efficiently for all-in-one low-level image restoration remains a challenge. Crucially, the field lacks an understanding of how VLMs organize hidden-layer representations and whether these structurally distinct paradigms share a common geometric organization for pixel-level perception. Such shared organization is a prerequisite for building highly transferable, unified restoration VLMs and adapters. In this paper, we systematically investigate representational similarity across 24 low-level tasks spanning 5 categories. We propose GeoSim, a unified four-level framework that analyzes task-conditioned representations from global similarity, local geometry, sparse feature decomposition, and topological verification perspectives. Our formulation applies to the analysis of hidden states in AR models and feature maps in DiTs across same- and cross-task/model settings. Our results reveal the organizing principles of low-level visual representations while exposing their limits in cross-task and cross-model agreement. Ultimately, GeoSim provides an interpretability lens for probing latent transferability in low-level vision and diagnosing model limitations in task- or model-specific scenarios.

0.32

STAR: A Spatial-Topology Aware Routing Framework for Generalizable 3D Scene Understanding

cs.CV ·2026-08-12 ·Mingwei Xing, Xinliang Wang, Yifeng Shi
code/data similarity 0.57
Abstract · The third author is the corresponding author

Constructing a unified 3D scene understanding model has long been hindered by the topological discrepancies across sensor modalities. While applying the Mixture-of-Experts (MoE) architecture is a flexible approach for multi-domain 3D understanding, we observe that conventional feature-only MoE routers may underrepresent local sampling topology under semantic supervision, making expert allocation difficult when semantic consistency coexists with geometric heterogeneity. To overcome this challenge, we propose STAR (Spatial-Topology Aware Routing Framework). Specifically, we introduce a multi-attribute self-supervised pre-training branch, covering topological and textural variations, to anchor cross-domain structural priors. Building upon this, we design a domain-aware expert branch with two mechanisms: Domain-Spatial-Guided Routing (DSR), which captures local topological variations from spatial context, and Entropy-controlled Dynamic Allocation (EDA), which adjusts the number of activated experts according to routing uncertainty. Together, these branches combine stable cross-domain representation learning with adaptive expert allocation. Extensive experiments across various tasks, encompassing both indoor and outdoor scenes, demonstrate the effectiveness of STAR. It achieves 80.1% mIoU on the ScanNet validation set and 77.2% mIoU on S3DIS, consistently improving over strong baselines. Code is available at our project page (https://xmw666.github.io/STAR/).

0.20

More Accurate, Less Human: Gestalt Grouping in Vision Models

cs.CV cs.LG ·2026-08-10 ·Sudhanva Manjunath Athreya, Sai Phani Kumar Malladi
benchmark similarity 0.56
Abstract · Accepted to VISxVision 2026, a workshop at IEEE VIS 2026. Includes appendix with per-task stimuli, metric derivations, and full per-model re…

Human vision organizes what it sees into wholes: same-colored points group into series, similar marks cohere into categories, and shapes complete into recognizable objects. These are the Gestalt operations that visualization design builds on. Whether vision models organize visual content this way has not been systematically tested. We introduce a behavioral battery that scores models against human data from prior perception studies on four grouping tasks: mark-color odd-one-out, color-series counting, silhouette recognition, and object odd-one-out. We apply it to 45 models across five training families: supervised, self-supervised, and contrastive vision-language encoders, open-weight VLMs, and closed foundation models. The battery reveals that agreement with human responses captures aspects of perceptual organization that conventional performance metrics fail to distinguish, with several closed models exhibiting substantially lower alignment than their benchmark accuracy would suggest. Scoring against published perception data therefore gives visualization research a reusable yardstick, requiring no new user study, for auditing whether the models now entering visualization pipelines organize what they see the way their human audience does.

0.13

Semantic-Spatial Discriminability Enhancement for Generalized Visual Grounding

cs.CV ·2026-08-31 ·Kaiyan Lei, Xu-Yao Zhang
top venue similarity 0.56
Abstract · Accepted by ACM MM 2026

Generalized Visual Grounding (GVG) task aims to localize targets in an image based on referring expressions, extends the classical visual grounding paradigm by integrating multi-target and non-target scenarios. Previous methods typically rely on global semantic matching or coarse-grained region interactions for localization, where the discriminative cues are primarily derived from sentence-level semantics or regional context. In complex multi-target scenarios, such approaches tend to confuse visually similar targets, making it difficult to establish stable instance-level decision boundaries. To address these limitations, this paper proposes a novel Semantic-Spatial Discriminability Enhancement (SSDE) framework for generalized visual grounding, which aims to enhance the discriminative ability on fine-grained semantics and spatial localization, improving both cross-modal understanding and instance-level grounding. Specifically, to enhance the semantic discriminability of query representations at the fine-grained level, we propose a Semantic Discriminability Enhancement (SeDE) module, which leverages spatially guided cross-attention to disentangle fine-grained target-relevant visual attributes and integrates them with the textual subject semantics. Furthermore, to strengthen the spatial discriminability of the referred targets, we introduce a Spatial Discriminability Enhancement (SpDE) module, which models an instance center density map to characterize the spatial distribution of targets, and explicitly constructs instance separation structures in the spatial domain by employing them as an auxiliary supervision signal. Extensive experiments show that SSDE achieves superior performance on ten datasets across both classic and generalized visual grounding tasks.

0.36

OneScene-Bench: Do Vision-Language Models Integrate Evidence Across Views?

cs.CV ·2026-10-28 ·Swathi Suhas, Anurag Das, Bernt Schiele, Jonas Fischer
publishedbenchmarkcode/data similarity 0.56
Abstract · Accepted at LM4RWD Workshop, Conference on Neural Information Processing Systems (NeurIPS), 2026

Vision-language models (VLMs) increasingly process multiple views of the same scene, but it remains unclear whether they integrate these observations into a coherent scene-level representation. Existing multi-image evaluations do not isolate this capability, as tasks may admit correct solutions through independent per-view processing without requiring cross-view correspondence. We introduce OneScene-Bench, a controlled benchmark designed to explicitly distinguish when cross-view integration is necessary. Our benchmark evaluates three complementary capabilities on real-world multi-view scenes: unique object counting, cross-view consistency, and spatial reasoning. For counting, we control object overlap across views to separate cases solvable by simple per-view aggregation from those requiring instance-level correspondence. For consistency, we introduce localized edits in shared and view-specific regions, testing whether models distinguish genuine scene contradictions from differences caused by viewpoint and visibility. For spatial reasoning, we construct object relations that require combining spatial evidence across viewpoints to compare physical proximity. We evaluate twelve state-of-the-art open-weight and proprietary VLMs and find that counting accuracy drops substantially on complementary views, ranging from 4.5-50.0%, compared with 45.5-81.5% on complete-overlap examples. Moreover, cross-view consistency remains around 50% across models, including recent proprietary VLMs, despite substantially stronger performance on spatial reasoning. Our results reveal a substantial gap between processing multiple images and integrating evidence across them, highlighting the need for evaluations that explicitly test scene-level multi-view understanding. Code and data are available at https://github.com/swathisuhas/OneScene-Bench.

0.30

Rethinking Feature Perturbation for Single-Source Domain Generalization: A Representation-Space Analysis and Spectral Case Study

cs.CV ·2026-10-22 ·Badr-Eddine Benelmostafa, Sohaib Baroud
similarity 0.55
Abstract

Feature perturbation is widely used in single-source domain generalization (SSDG), yet it is typically evaluated by task performance rather than by the representation changes it induces. We investigate whether perturbations can be compared through measurable descriptors of these changes. We introduce a four-dimensional representation-space description comprising feature displacement $D_F$, relational displacement $D_R$, a task-region ratio $Θ_{\mathrm{Task}}$, and subspace deviation $Θ_{\mathrm{Subspace}}$. We use these descriptors as an empirical analysis framework, not as a universal geometry, to characterize existing perturbation mechanisms. Across two object-detection settings involving style and weather shifts, we observe distinct geometric profiles and a concentration of relatively strong out-of-domain (OOD) configurations within a bounded region of the descriptor space. These findings do not establish a universal optimum but motivate investigating whether regions associated with stronger SSDG performance can be identified without target-domain access. As a case study, we introduce Dual-Invariant Spectral Perturbation (DISP), a non-learned Fourier-phase perturbation with explicit controls over phase magnitude $ρ$, channel coupling $κ$, and frequency support $\mathcal{M}$. We empirically examine how these controls affect the proposed descriptors. DISP achieves strong OOD performance in our experiments, illustrating how explicit perturbation controls can be analyzed through a representation-space description. We hope this viewpoint can contribute to more interpretable, controllable, and ultimately testable approaches to feature perturbation in SSDG.

0.23

Guided Group Featurizers (GGF): Improving Block-Sparse Featurizers with Object-Guided Learning

cs.CV ·2026-11-13 ·Waldemar Chang, Andrii Krutsylo
similarity 0.54
Abstract

Block-sparse featurizers (BSFs) use groups of learned directions to capture the multidimensional structure of visual concepts. However, grouping alone does not ensure that information about a concept is consolidated within a single group. We introduce Guided Group Featurizers (GGF), a novel object-guided extension that uses object annotations for initialization and training while preserving the BSF architecture. We assess whether each group's strongest response regions share an object type, part or material across images. Blinded GPT reviews give GGF groups an average semantic coherence score of $93.4\pm1.6$%, versus $83.4\pm2.5$% for SLiCS and $67.9\pm3.1$% for supervised Vanilla BSF. GGF also leads in a separate blinded Gemini review. GGF's consolidation advantage extends to unguided categories: one selected group per category covers $66.1\pm1.9$% of target-containing crops, versus $59.2\pm1.8$% for supervised Vanilla BSF, at approximately 5% false-positive rates for both. Across COCO and VOC, GGF also improves single-group object Capture and localization over Vanilla BSFs, SLiCS and Concept-SAE. Its groups produce more focused and reproducible object responses while supporting recognition and retaining associations under rotation, scale and brightness changes. GGF also uses fewer learned parameters than Concept-SAE and shorter recorded VOC training loops than both supervised alternatives. These findings support object guidance for organizing more interpretable BSF groups and show that response concentration extends to categories excluded from guidance.

0.23

ReCoCo: From Patches to Compositions for Visual Representation Learning

cs.CV ·2026-11-18 ·Juan Gutiérrez, Saman Zadtootaghaj, José Luis Blanco-Murillo, Sara Atito
similarity 0.54
Abstract

Self-supervised learning has produced powerful visual representations, yet learning strong dense features remains challenging. While consistency between augmented views provides strong image-level supervision, extending this principle to local visual content is less straightforward. Existing approaches often match individual patches across views or identify coherent regions, introducing uncertainties in correspondence estimation and assumptions about how visual content should be grouped. We introduce ReCoCo (Regional Composition Consistency), a principled approach that extends cross-view consistency to the regional level without requiring such correspondences or semantic grouping. ReCoCo represents the content shared between augmented views through compositions of their patch-level predictions over learned visual prototypes. By aligning these compositions, the model learns from regions containing multiple visual concepts without requiring them to be separated beforehand. ReCoCo integrates naturally with teacher-student self-distillation and introduces no additional learned parameters. Extensive experiments demonstrate substantial improvements in frozen-feature segmentation, clustering, and regional retrieval while maintaining competitive image-level recognition, alleviating the trade-off between dense and global representation quality.

0.21