CS Paper Radar

Partition, Refine, and Detect: Preserving Novel Category Capacity in Open-Vocabulary Object Detection

Wei Zhang, Zhu Teng, Zhixiang Chi, Junnan Zou, Yi Liu, Jianping Fan

cs.CV cs.LG ·submitted 2026-11-08 04:13 UTC · v1
top venue

Abstract

Open-vocabulary object detection (OVD) expects the model trained on base classes to recognize both base and novel classes at inference. Typical methods rely on CLIP-style vision-language embeddings, yet the learned space is often biased toward base categories, with limited effective capacity preserved for novel concepts. Such a base-dominated representation can compress novel semantics, leading to semantic collapse and mis-calibrated confidence at inference. To address this issue, we present PR-OVD, a partition-and-refine framework that explicitly reserves capacity for novel categories and strengthens category separation, while being seamlessly integrated into different detector architectures. Specifically, PR-OVD first performs coarse partitioning to build global divisions for base, novel, and an explicit background partition in both visual and textual modalities, preventing object-irrelevant embeddings from leaking into category-specific representations. It then applies category-level refinement within each partition to sharpen decision boundaries and alleviate the lack of fine-grained region-level pretraining in CLIP. To further provide reliable visual anchors for novel categories, we design a cognitively inspired Structure Prompt (SP) that encodes both egocentric attributes and allocentric context, facilitating stable partition-and-refine learning. Extensive experiments on two OVD benchmarks show that PR-OVD delivers an average 6.5\% proportional improvement on novel categories over the state-of-the-art methods. Notably, PR-OVD also benefits other open-vocabulary tasks, including segmentation and classification, further confirming its effectiveness and generality.

Comment
IEEE Transactions on Multimedia (TMM)

Closest papers

Nearest neighbours by meaning across the whole index.

InterHier: Learning Interconnected Hierarchical Semantics for Open-Vocabulary Object Detection

cs.CV cs.AI ·2026-09-21 ·Yeong-Jin Kim, Ho-Joong Kim, Seong-Whan Lee
published similarity 0.77
Abstract · 12 pages, 6 figures. Published in IEEE Access

In this paper, we investigate the limitations of fixed, hand-crafted connectors in hierarchical semantic representations for open-vocabulary object detection. Existing methods establish semantic relationships between base categories and unseen novel categories by placing a fixed connector between adjacent super-/sub-categories. However, such fixed connectors may not optimally capture the relationships within a semantic hierarchy. To address this limitation, we propose interconnected hierarchical semantic representations (InterHier), which utilize a prepended learnable context to globally guide the interpretation of prompts containing hierarchical relationships. InterHier operates in two main stages. First, it constructs a hierarchy-aware prompt by integrating super-/sub-categories and prepending a learnable context. Second, it optimizes this learnable context to align visual region embeddings and textual embeddings. InterHier consistently improves performance over methods that rely on fixed connectors and can be seamlessly integrated into existing open-vocabulary object detection models. Experiments on open-vocabulary object detection benchmarks demonstrate that InterHier achieves competitive performance against state-of-the-art methods.

0.27

FocusOVD: Geometric and Contextual Feature Refinement for Open-Vocabulary Small-Object Detection

cs.CV ·2026-10-20 ·Hanlin Kou, Beiqi Bai, Li Chen, Youjia Jing, Shupan Li +1 more
published similarity 0.72
Abstract · Accepted as a full paper at ChineseCSCW 2026

Open-vocabulary object detection aims to detect both seen and unseen categories by leveraging the semantic knowledge of vision-language models. However, existing methods often struggle with small objects due to insufficient preservation of high-resolution details, ineffective information aggregation across feature levels, and limited region-level discriminability. To address these challenges, we propose FocusOVD, an open-vocabulary object detector that enhances small-object representations without modifying the frozen VLM backbone. FocusOVD introduces three complementary components: a Reweighted Cross-scale Feature Fusion module, which learns normalized level-wise weights to adaptively balance shallow appearance details and deep semantic cues during feature pyramid aggregation; a Residual Pinwheel-shaped Gated Convolution module, which strengthens local geometric responses in high-resolution features; and a Residual Convolutional Gated Linear Unit module, which suppresses background-dominated activations and enhances RoI discriminability before prediction. Extensive experiments on COCO-OVD and LVIS-OVD demonstrate the effectiveness of FocusOVD. On COCO-OVD, FocusOVD achieves a novel-category AP of 36.3%, outperforming CORA, UniDetector, and SAMP by 1.3%, 1.1%, and 1.5%, respectively; on LVIS-OVD, it achieves an overall AP of 33.7%, surpassing the strongest competing method by 0.6%. These results demonstrate the strong generalization ability of FocusOVD for open-vocabulary small-object detection.

0.27

RT-DETR-World: Transferring Rich LLM Semantics to Real-Time Open-Vocabulary Detection

cs.CV ·2026-10-07 ·Yupeng Zhang, Ziyi Zhao, Juntao Cheng, Sheng Wang, Ningnan Guo +2 more
similarity 0.71
Abstract

Open-vocabulary detection (OVD) recognizes categories unseen during training through textual category queries, yet achieving strong generalization with real-time efficiency remains challenging. Beyond vocabulary scaling, zero-shot generalization may benefit from reusable visual--semantic cues learned from seen data, including attributes, actions, states, and contextual relations. Existing real-time OVD methods primarily emphasize vocabulary coverage and efficient region/query--text matching; under strict efficiency constraints, compact detectors may struggle to absorb rich instance semantics and scene context. We propose RT-DETR-World, a compact DETR-style detector that transfers the rich semantics conveyed by descriptions during training while retaining lightweight query--text matching at inference. We construct GroundingCapv2 with three levels of supervision: category names for standard OVD, object descriptions conveying instance-level semantics, and image descriptions conveying object relations and scene context. These descriptions serve only as training-time semantic supervision. To help the compact detector absorb these semantics, we propose Dual-Path Description Alignment (DDA), combining a deployment-consistent MiniLM pathway with a training-only LLM teacher. MiniLM provides query--category supervision and object-description alignment, while offline teacher features supervise matched queries and global visual representations at the object and image levels, respectively. All teacher features are precomputed, and the teacher-side modules are removed after training. We further propose Relation-Aware Negative Relaxation (RNR), which uses teacher-derived semantic similarities to relax related negatives while preserving exact positives. Experiments demonstrate competitive zero-shot accuracy and a favorable accuracy--efficiency trade-off. The code will be released.

0.19

What Remains OOD? OODConceptBench for Open-World Concept Recognition in Vision-Language Models

cs.CV ·2026-10-24 ·Zhiran Luo, Zhiyong Yang, Zitai Wang, Sicong Li, Qianqian Xu +1 more
benchmark similarity 0.69
Abstract

In the era of large-scale pretrained models, zero-shot generalization has increasingly blurred the boundary between known and unknown concepts. However, current evaluations do not directly reveal which open-world concepts different vision-language models can recognize. To study this problem, we introduce OODConceptBench, a hierarchical benchmark for open-world concept recognition on image-concept pairs. The full benchmark set (L1) contains 30,752 images, and the harder subset (L2) contains 14,285 images. The benchmark covers 560 concept phrases across seven open-world categories. We evaluate 37 vision-language models across release year, model family, and variant. We obtain three main findings. First, target-concept recognition is lower on L2 than on L1 across all 37 evaluated models. Second, later-released models do not necessarily achieve stronger open-world concept recognition. Third, an increase in overall recognition does not imply consistent improvement across all benchmark categories. Together, these results show that OOD remains a challenge even in the era of large-scale vision-language pretraining.

0.25

CorrCLIPv2: Harnessing Unlabeled Images for Open-Vocabulary Semantic Segmentation

cs.CV ·2026-10-17 ·Dengke Zhang, Fagui Liu, Quan Tang, C. L. Philip Chen
top venuecode/data similarity 0.68
Abstract · 25 pages, 12 figures, 12 tables. Published in International Journal of Computer Vision

Open-vocabulary semantic segmentation aims to classify pixels without being restricted to a fixed set of categories. Although Contrastive Language-Image Pre-training (CLIP) demonstrates remarkable potential for open-vocabulary recognition, its segmentation capability is impaired by noisy inter-class patch interactions. To this end, this work proposes a unified training-free object-centric framework, named CorrCLIPv2, from three complementary aspects of patch correlation, feature representation, and output regularization. Specifically, for patch correlation, we leverage class-agnostic mask generators to explicitly disentangle inter-class interactions. Moreover, we utilize spatially-coherent features from self-supervised models to compute attention scores to mitigate mask inaccuracies. Regarding feature representation, we devise a refinement mechanism to simultaneously strengthen spatial details and semantic representation. Within the output space, we unify predictions via voting in each mask to enhance spatial consistency. To mitigate contextual bias and enhance prediction accuracy of objects, we further extend the framework by incorporating a cross-image object retrieval paradigm, which incurs minimal computational overhead. We build a memory bank of CorrCLIP-enhanced features from unlabeled images and implement a regime of multi-CLIP ensemble. By aggregating semantically consistent object features across diverse contexts retrieved from the memory bank, CorrCLIPv2 mitigates context-induced interference and obtains more robust predictions. Extensive experiments demonstrate that our approach establishes a new state-of-the-art on 10 standard benchmarks. Remarkably, it even surpasses fully supervised open-vocabulary methods across 22 multi-domain datasets. Code is available at: https://github.com/zdk258/CorrCLIP.

0.44

Reprogramming Vision-Language Models via Structured Prompt Reparameterization

cs.CV ·2026-09-29 ·Zizhao Li, Chengyi Cai, Mohammed Yaqoob Ansari, Feng Liu, Joseph West +1 more
similarity 0.68
Abstract

Visual reprogramming adapts pretrained models to downstream tasks by modifying their input and output interfaces while keeping the backbone fixed. In vision-language models, existing methods mainly rely on intra-class prompt aggregation and do not explicitly model relationships among classes. However, fine-grained categories often exhibit highly overlapping attribute descriptions and strong inter-class correlation in the text embedding space, where discriminative cues lie in subtle low-variance components. We propose Reparameterized Inter-Class Visual Reprogramming (RVP), a structured framework that aggregates multiple text prompts within each class and applies residual correction across classes. We also show that CLIP-based visual reprogramming with input-independent linear output aggregation can be expressed as a linear mapping from frozen image embeddings to downstream logits, and use this view to design a structured reparameterization that models shared semantic components and class-specific differences. RVP uses only a single visual prompt and can be reparameterized at inference into a frozen backbone followed by a linear classifier, incurring nearly zero computational overhead. Across 11 few-shot classification benchmarks and four CLIP backbones, RVP consistently improves over prior visual reprogramming methods with comparable or better inference efficiency.

0.18

UniCounting: Instance-Aware Proposal Consolidation for Image-Query-Free Multi-Category Counting

cs.CV ·2026-10-06 ·Jinshi Liu, Pan Liu, Lei He, Weichao Luo, Rui Qian
similarity 0.68
Abstract

Visual counting is commonly formulated as counting a single specified target, with a model receiving an image-specific exemplar, text query, or target category and returning a single count. We instead study fixed-vocabulary image-query-free multi-category counting. A global vocabulary is fixed for each run, and, given only an RGB image, the model predicts a complete category--count vector without being told which categories appear. We present UniCounting, which casts counting as instance-aware structural inference over an over-complete proposal set. Generic segmenters produce duplicate masks, partial views, and proposals from neighboring instances; semantic scores can name them but cannot determine which denote the same object. Frozen SAM~2.1 generates masks, while frozen DINOv2 and OpenCLIP provide relation and category features. A 3,267-parameter category-shared relation head predicts same-instance affinities from instance-mask-derived supervision. Sparse graph construction, representative selection, labeling, and background-margin admission then convert each admitted component into one count with replayable group evidence. Only the relation head is trained, without count or density-map targets. On COCO clean500, UniCounting obtains lower point-estimate vector $\ell_1$ error and absent-class false mass than calibrated OWLv2-All80, with comparable micro presence F1. Under a matched decoder, the learned relation reduces both errors relative to mask containment, mask IoU, CLIP, and DINO, while revealing a fragmentation--merge trade-off. We also report transfer diagnostics on OmniCount-sub, FSC-147, and CARPK.

0.13

Vague2Detect: Handling Ambiguous Prompts in Knowledge-Based Open-World Detection

cs.CV cs.CLcs.LG ·2026-09-09 ·Ibrohimjon Muminov, Jihie Kim
benchmarkcode/data similarity 0.68
Abstract · 15 pages, 4 figures, 3 tables. Code: https://github.com/ibrohimgets/Vague2Detect

Real-world detectors must often interpret functional or ambiguous prompts, yet conventional models such as YOLO remain restricted to fixed class lists. Even open-vocabulary models like YOLO-World frequently misalign vague language with the intended objects. Building on our prior work Commonsense-Guided Open-World Object Detection Using LLMs and Visual-Semantic Matching, we address YOLO-World's limitations in grounding task-driven queries. We propose Vague2Detect, a hybrid pipeline in which a fine-tuned Sentence-BERT retrieves candidates from a structured household Knowledge Base (KB), and YOLO-World verifies their presence in the image. For prompts outside the KB, a large language model (GPT-3.5-turbo) generates candidate descriptions, dynamically expanding the KB to cover novel concepts. On a benchmark of household scenes using custom images and an Open Images V7 subset, YOLO-World alone achieves only 32% Vague Prompt Success Rate (VPSR), the ability to map ambiguous queries to correct detections. In contrast, Vague2Detect improves performance to 61% VPSR with high precision, and up to 85% when augmented with GPT fallback.

0.30