CS Paper Radar

Cost-Aware Vision--Language Model Arbitration for Fabric Structure Recognition A Deployable Multi-Agent System

Chenwei Wang, Haochen Li, Shuk Ching Tang, Misbah Iqbal, Carman Lee, Elif Ozden-Yenigun

eess.IV ·submitted 2026-09-09 11:40 UTC · v1
benchmark

Abstract

Recognizing a fabric's structure is a prerequisite for translating textile-specific material information into structured digital form for downstream supply-chain systems. Pure CNN classifiers are cost-efficient but fail on visually ambiguous categories; vision--language models (VLMs) generalize more broadly but cost much more per image and are unstable on specialist domains. We present a multi-agent system in which a CNN cascade handles the easy majority and a VLM is invoked only as a selective arbiter, constrained to a top-3 taxonomy-consistent choice. The fabric taxonomy performs as a constraint for the whole recognition process to increase the accuracy and reduce the VLM calls. Meanwhile, the CNN cascade is distilled to a small parameter size to reduce the inference time and meet the needs of practical deployment. On a newly curated 14-class benchmark, a flat ConvNeXt-Tiny baseline reaches $90.45\,\%$ top-1 and $76.9\,\%$ on the four hardest classes; \method's hierarchical cascade reaches $93.94\,\%$ top-1 and $94.50\,\%$ hard ($+17.6$\,pp). Tightening the VLM trigger from $60\,\%$ to $<\!10\,\%$ cuts API cost by ${\sim}90\,\%$ with no measurable accuracy loss. CPU inference is $\le\!93$\,ms without a VLM call ($9.3$\,ms distilled). Each prediction carries a machine-readable reasoning record, offered as an entry point for future supply-chain documentation.

Closest papers

Nearest neighbours by meaning across the whole index.

From Benchmark to Deployment: Shift-Robust Fabric Recognition for Industrial Textile Onboarding

eess.SY ·2026-09-12 ·Haochen Li, Chenwei Wang, Felicity S. C. Tang, Misbah Iqbal, Carman K. M. Lee +1 more
benchmark similarity 0.65
Abstract

Automatically recognising a fabric's construction (jersey, twill, satin) is a bottleneck in textile sourcing, where incoming swatches are still typed by hand. Benchmark accuracy suggests the problem is solved, yet rarely survives deployment. On the \numClasses{}-class FabricFlow benchmark we expose three gaps that headline accuracy hides. First, a duplication audit reveals train/test leakage that inflates accuracy; we rebuild leakage-free splits that report the true difficulty. Second, on the clean data the binding failure is acquisition-source shift between catalogues, not the peripheral shortcuts one might fear: on an archive-exclusive hold-out, standard training holds 58.0\% Top-1 at a calibration error of 0.158, while a simple, architecture-agnostic central-texture recipe adds 13.5 Top-1 points and restores calibration. Third, because confusing one fabric family for another is costlier than a within-family slip, we optimise a taxonomic-severity cost: a confidence-gated routing policy auto-types confident swatches and refers only the uncertain minority to a human, sharply cutting onboarding cost. Throughout we report honest negatives: hierarchical classification, OCR fusion and zero-shot vision--language models all fail to help, yielding a concrete, calibrated, cost-aware recipe for deployable textile onboarding.

0.27

Carve VFM: Carving Specialists out of Generalists, Automatic Task-Specific Compression of Vision Foundation Models

cs.CV cs.LG ·2026-10-12 ·Lotfi Abdelkrim Mecharbat, Abdelrahman Elsayed, Ibrahim Almakky, Martin Takáč, Mohammad Yaqub
similarity 0.63
Abstract · 24 pages, 6 figures, 11 tables

Vision foundation models (VFMs) provide transferable visual representations, but their inference cost limits deployment in many settings. Fine-tuning specializes a VFM without shrinking it; distillation shrinks it, but through expensive teacher-guided training. Structured pruning offers a more direct route by extracting a compact subnetwork from the VFM. However, as compression gets aggressive, existing methods fail because they treat subnetwork selection and recovery as separate axes. Across layers, capacity must follow task sensitivity rather than spread uniformly. Within a layer, a removal cannot be judged by importance alone; its impact also depends on how well the lost contribution can be recovered from what remains. Selection and recovery must therefore be decided jointly. We introduce Carve VFM, which resolves this interdependence. Given a VFM, a calibration set, and a parameter budget, it (i) searches for task-specific capacity allocations across layers; (ii) guides the search via an adaptive recovery-aware fitness criterion; and (iii) reconstructs removed behavior through iterative closed-form low-rank corrections merged into the surviving weights. The result is a dense encoder produced in minutes on a single GPU without backbone retraining. Across twelve benchmarks and four models from DINOv3 and MetaCLIP-2, Carve VFM achieves the highest reported mean accuracy in seven of eight backbone-budget configurations among the evaluated dense structured-pruning methods. Further fine-tuning surpasses the strongest evaluated edge model (88.1% vs. 85.8%), and carved models cut compute and memory while delivering 1.3-2.0x higher GPU throughput than the uncompressed one. These results show that the compact model you need is inside the VFM; it only has to be carved out.

0.22

ConvCue: Complementary Visual Inductive Biases for Vision-Language Models

cs.CV ·2026-09-28 ·Zixuan Lan, Shichu Sun
benchmark similarity 0.63
Abstract

Modern vision-language models (VLMs) achieve strong performance across a broad range of multimodal tasks, yet still struggle with visual questions that require fine-grained discrimination and spatial understanding. These limitations motivate investigating whether supplementary visual representations can improve existing VLMs without replacing their native visual encoders. Pretrained convolutional networks offer a candidate feature source, motivated by their local connectivity and spatial weight sharing. We introduce CONVCUE, which augments the native visual representations of a pretrained VLM with final-stage features from a parallel, frozen pretrained CNN. A learnable adapter maps convolutional features to the native visual feature dimension, while gated cross-attention allows the original visual tokens to retrieve information from the CNN features. The enhanced tokens are passed through the original visual-to-language projector, and the model is adapted through a two-stage training procedure. We evaluate CONVCUE on Qwen3-VL-2B, Qwen3-VL-4B, and LLaVA-OneVision-7B across 13 multimodal benchmarks covering visual question answering, document and chart understanding, and multimodal reasoning. CONVCUE improves average benchmark performance over both the original models and matched two-stage fine-tuning controls on all three backbones. On Qwen3-VL-4B, it improves over the original model on all 13 benchmarks and raises the average score from 75.00 to 78.82 relative to the matched fine-tuning control. These results show that pretrained convolutional representations, when integrated through learned adaptation and fusion, can improve the visual understanding of existing VLMs without replacing their original visual encoders.

0.19

PRISM-Net: Dynamic Routing Network with VLM-Guided Disentangled Attribute Distillation for Explainable Fine-Grained Product Retrieval

cs.CV ·2026-10-30 ·Chung-Ming Lo, Che-Wei Chang
similarity 0.62
Abstract · 10 pages

Fine-grained visual product retrieval in large-scale e-commerce multimedia systems demands discriminative representations capable of simultaneously resolving macro-level category semantics and micro-level visual attributes such as structural contour (line) and chromatic appearance (color). However, conventional backbones including convolutional neural networks (CNNs) and vision Transformers (ViTs) inherently suffer from feature entanglement and shortcut learning. To learn the micro-level visual attributes without costly manual attribute annotations, we propose phase-routed interpretable semantic mixture-of-experts network (PRISM-Net), an explainable multi-expert architecture that disentangles holistic visual features into disentangled line and color expert pathways governed by a conditional meta-router. In phase 1, we introduce an external VLM-Guided sigmoid semantic distillation mechanism with gradient-detached routing gates, utilizing zero-shot text prompts from SigLIP to supervise specialized structural and chromatic experts. In phase 2, the disentangled expert embeddings are adaptively fused with macro base features via a learned meta-router gate to calibrate category boundaries. Extensive 5-fold cross-validation across 275,454 e-commerce images (18 categories) and full-scale retrieval benchmarks (55,091 queries against a 220,363-image gallery) across SwinV2, ConvNeXt-V2, and DINOv3 demonstrate substantial gains. On SwinV2, PRISM-Net boosts full-fold product mAP@10 from 80.98% to 83.90% (+2.92%), line mAP@10 by +5.74%, and color mAP@10 by +9.12%. Furthermore, unsupervised distribution analysis of the meta-router gate reveals that both ViT and CNN backbones emergently allocate up to 6.8x higher structural line weights to rigid appliances and hardware while dedicating >96% chromatic weight to clothing.

0.31

ConvRoute-ViT: Convolutional Token Routing for Efficient Vision Transformers

cs.CV ·2026-10-28 ·Manish Das, Shiv Ram Dubey, Satish Kumar Singh
similarity 0.62
Abstract · 5 pages, Conference

Vision Transformers (ViTs) achieve strong image recognition performance, but their self-attention increases quadratically with the number of input tokens and makes them computationally expensive. The standard implementation of ViT models treats all patch tokens with equal importance across every layer of the model. This uniform treatment introduces redundant computation and may reduce the model's focus on the most discriminative object regions. In this paper, we propose ConvRoute-ViT, a convolutional token routing mechanism for efficient vision transformers that performs token reduction through convolutional routing. Our model preserves the most relevant patch tokens according to class-token attention at selected layers, while the remaining tokens are spatially reorganized and compressed into a compact set of convolutional summary tokens. This design reduces the sequence length without completely removing the contextual information of the less informative tokens. By applying the routing process hierarchically, the proposed framework constructs compact and discriminative representations for deeper layers. Experiments on CIFAR-100 and TinyImageNet show that ConvRoute-ViT improves computational efficiency while maintaining classification accuracy compared with the base ViT.

0.16

MLLM-Routed Heterogeneous Ensembles for Robust Cross-Dataset Image Classification

cs.CV cs.AIcs.CLcs.LG ·2026-08-13 ·Daniel Perkins, John Squires, Janou Milligan, Chandra Raskoti, Linda Ungerboeck
similarity 0.62
Abstract · 15 pages (single column), 4 figures, 7 tables

Modern image classification models excel when trained on single task-specific datasets but often struggle to generalize across domains and difficulty levels. We propose ARMDIL, an Adaptive Router for Multi-Domain Image Classification with LLMs. ARMDIL is an ensemble that uses a multimodal large language model (MLLM) agent to dynamically route each image to the most suitable vision backbone. Our diverse ensemble employs convolutional neural networks (ResNets), self-supervised representation learners (SSL), and vision language models (VLMs), each trained on a unified label space constructed from multiple image datasets with differing distributions and characteristics. Empirical evaluations illuminate the distinct capabilities and vulnerabilities of each architecture across disparate visual domains. Crucially, we show that ARMDIL effectively navigates these tradeoffs, performing competitively with specialized training-based routers. Furthermore, it drastically improves adaptability by allowing new information to be integrated via simple prompt modifications, while enhancing interpretability through natural language reasoning traces. These advances in cross-dataset image classification pave the way for more reliable general-purpose vision systems such as AI assistants and autonomous robots.

0.10

Rethinking Data Efficiency in Industrial Dense Prediction: Pretraining Coherence, Not Inductive Bias, Determines ViTs Low-Data Advantage

cs.CV ·2026-08-11 ·Haoran Sui, Yaoyuan Jia
similarity 0.62
Abstract · 14 pages, 10 figures, 17 tables. This paper targets industrial defect detection via vision transformer and CNN alignment grafting

Vision Transformers (ViTs) are widely believed to require more labeled data than CNNs for industrial dense prediction. Through controlled experiments on four industrial datasets, we show that the data-efficiency gap stems from pretraining incoherence, which refers to the statistical mismatch between ImageNet-pretrained ViT backbones and COCO-pretrained CNN necks, rather than from inherent self-attention deficits. We characterize the cross-architecture feature gap and propose a lightweight AlignBlock family for pyramid-level feature recalibration. Our core finding empirically identifies a data-efficiency frontier: for domain-proximal scenes with >= 200 samples, Swin-Graft surpasses YOLOv11x (terminal 703-shot: 0.973 vs 0.956 mAP@50); for domain-distant scenes, CNNs retain advantage (hook 141-shot: 0.900 vs 0.600 mAP@50). Grafted neck weights yield up to 2.5x the mAP of a randomly initialized neck.

0.10

Not All Error Yields to Scale: Where Scaling Stops in Vision-Language Inference

cs.CV cs.AI ·2026-10-01 ·Xinye Zhao, Yunkai Dang, Yunchen Wu, Wenbin Li
similarity 0.62
Abstract

Vision-language models (VLMs) face a fixed-budget trade-off between processing more visual information for fine-grained perception and using a larger language backbone for complex reasoning. Existing studies do not tell us which combination of backbone size and input resolution to deploy, especially in high-resolution deployments. To address this gap, we propose the Separable Law that describes how VLM performance changes with language backbone size and visual token count. We fit the law to measurements from 26 InternVL and QwenVL models, with language backbone sizes from 1B to 72B, on four high-resolution benchmarks with image sizes from 224 pixels to 8K. We find that the questions responding to scaling can be predicted from the skill they require, while a substantial fraction never responds at all. We also find that the two model families gain similarly from a larger backbone, while their gains from more visual tokens differ sharply. Combined with a cost law, the Separable Law gives a closed-form rule for allocating compute between backbone size and visual tokens. When deployment is limited to available configurations, the law identifies model and image sizes that perform close to the best feasible choice under the same budget. We hope our work offers a principled way to decide how much a model should be allowed to see at high resolution, given what it must reason about.

0.43