CS Paper Radar

A VLM Answer Is Not an Anomaly Score: Rank Compression Across Image and Video Anomaly Detection

Inpyo Song, Jangwon Lee

cs.CV ·submitted 2026-08-21 15:56 UTC · v3
benchmark

Abstract

Anomaly detection aims to identify observations that deviate from normal patterns. Recent work has increasingly used pretrained vision--language models (VLMs) for training-free image and video anomaly detection without task-specific retraining. Anomaly detection is commonly evaluated by how well anomaly scores rank anomalous images or video frames above normal ones. Generative VLMs, however, assign probabilities to possible answers and then decode a single answer. This decoding step may discard ordering information. We call this loss decoded-answer rank compression. To isolate this effect, we compare two ways of scoring the same VLM output: one uses only the decoded answer, while the other computes a probability-weighted score over all possible answers. Probability-weighted scoring consistently outperforms decoded-answer scoring across image and video anomaly detection benchmarks, using different VLMs and answer scales. The mean gains range from 7.66 to 19.95 points on the primary metrics. Most of this gap comes from decoded-answer ties. Breaking decoded-answer ties with answer probabilities recovers at least 95% of the average performance gap on every benchmark. We further investigate how these ties affect evaluation. We find that evaluating tied scores one input at a time makes the reported AUROC depend on input order, and that linear PR interpolation can inflate reported performance. These results reveal that both how VLM outputs are scored and how discrete anomaly scores are evaluated affect reported anomaly-detection performance.

Comment
Preprint

Closest papers

Nearest neighbours by meaning across the whole index.

Probe-VAD: Ordinal Likelihood Probing for Training-Free Video Anomaly Detection

cs.CV ·2026-09-15 ·Jiawei Gu, Qilin Zhao, Tengkuo Guo, Zhiming Zhong, Shuangqing Zhang +4 more
code/data similarity 0.75
Abstract · Under Review

Video anomaly detection (VAD) aims to localize anomalous events in untrimmed videos. Vision-language models (VLMs) provide rich visual understanding for training-free VAD, but existing approaches impose restrictive interfaces between visual understanding and anomaly scoring. Caption-based pipelines compress visual evidence into text, potentially discarding subtle cues, while direct numerical generation forces the model to express its judgment through a small set of predefined scores. Such interfaces can obscure subtle differences in anomaly severity, causing visually distinct clips to receive similar representations or scores and thereby limiting the resolution of anomaly ranking. We propose \textbf{Probe-VAD}, an ordinal binary-probing framework that directly probes severity preferences from a frozen VLM. Given raw video clips, Probe-VAD queries ten ordered severity thresholds and extracts constrained \textit{YES}/\textit{NO} continuation likelihoods. Their normalized preferences form a cumulative severity profile, from which tail evidence is aggregated into a continuous anomaly score, with isotonic projection enforcing ordinal consistency. Experiments on public VAD benchmarks demonstrate superior performance with low computational cost. Probe-VAD provides a simple interface for translating frozen VLM visual understanding into continuous, rank-sensitive anomaly scores without task-specific training or caption-based compression. Code is available at: https://github.com/yvestine/COVAS-VAD.

0.38

Anomaly-Guided Observation and Two-Level Semantic Feedback for Training-Free Video Anomaly Analysis

cs.CV ·2026-10-12 ·Hangil Park, Seungheon Kim, Tae-Kyun Kim
similarity 0.73
Abstract

Training-free language-based video anomaly detection (VAD) uses frozen vision--language models (VLMs) and large language models (LLMs) to localize abnormal events without task-specific model-weight optimization. Existing approaches increasingly exploit broader temporal and semantic context, yet generic context does not necessarily foreground abnormal events or specify how video-level semantics should modify local anomaly scores. We introduce a training-free semantic feedback framework that uses VAD-guided active observation for video anomaly understanding (VAU) and returns the resulting anomaly-focused description to refine temporal detection. Starting from an upstream anomaly-score curve, we adapt reasoning-based iterative frame selection: high-score regions seed the observations, and anomaly scores together with VLM relevance assessments guide subsequent temporal exploration. A frozen video-language model generates a single open-ended anomaly description from the selected frames. The description is returned through two complementary feedback levels: video-level feedback determines whether the score distribution should be suppressed, preserved, or sharpened, with selective visual rechecking, while local feedback retrieves event-related windows and promotes scores only after target-local visual verification. All model weights remain frozen, and feedback is applied once without regenerating the description. On the HIVAU video-summary evaluations of UCF-Crime and XD-Violence, VAD-guided observation improves GPT-rated reasonability, detail, and consistency over the matched CLIP-guided variant. For VAD, semantic feedback improves the re-evaluated upstream detector from 84.36% to 86.25% AUC on UCF-Crime and from 68.07% to 72.58% AP on XD-Violence, where the final AUC is 91.74%.

0.16

Parser-Free VLM Verification for Federated Weakly Supervised Video Anomaly Detection

cs.CV cs.AIcs.LG ·2026-09-07 ·Sébastien Thuau, Amira Gran, Siba Haidar, Rachid Chelouah
similarity 0.70
Abstract · 6 pages, 1 figure, AVSS 2026

How can vision-language models help video anomaly detection (VAD) when surveillance data remain distributed, weakly labeled, and resource-constrained? Most weakly supervised VAD methods assume centralized training; recent VLM-based extensions further rely on dense inference, generated explanations, or additional adaptation. We introduce a lightweight federated MIL-VLM cascade in which only a compact MIL scorer is trained across clients, while a frozen VLM verifies high-scoring suspect segments post hoc. We study two VLM feedback interfaces: parsed text-generation decisions and a logit-based interface that extracts a continuous anomaly score from next-token Yes/No probabilities. Experiments on UCF-Crime with InternVL3.5-2B and Qwen3-VL-2B-Instruct show that text-generation verification can improve frame-level AUC after diagnostic temporal post-processing, but remains sensitive to prompts, parsers, model choice, and smoothing. In contrast, the logit interface provides a fixed parser-free signal that improves both frame-level AUC and frame-level AP over the MIL baseline across both VLMs, without temporal post-processing in its main configuration. Since suspect segments are updated independently once available, next-token logit feedback provides a simple segment-local alternative to text-generation verification.

0.13

A Unified Score Matching Paradigm for Video Anomaly Detection and Anticipation

cs.CV eess.IV ·2026-10-08 ·Congqi Cao, Zhenhe Liang, Hanwen Zhang, Yifan Zhao, Qinyi Lv +2 more
benchmark similarity 0.70
Abstract

Video anomaly detection (VAD) is a fundamental and safety-critical task in computer vision. Recent generative approaches detect anomalies from a distributional perspective, but remain limited by local anomaly modes. Meanwhile, video anomaly anticipation (VAA), as a proactive extension beyond post-hoc detection, introduces additional challenges. In particular, the contrastive inference paradigm in VAD, which relies on ground-truth frames, is not applicable to VAA, hindering its development. To address these challenges, we propose a unified score-driven framework, termed Uni-DSM, based on denoising score matching (DSM), which models anomaly patterns through likelihood estimation and score functions over the learned data distribution. Within this unified framework, we adopt a shared noise-conditioned score transformer backbone with scene-dependent embeddings and motion-aware weighting for distribution-level modeling. Instead of introducing separate architectures, Uni-DSM unifies VAD and VAA through different inference and supervision paradigms built upon the same score-based formulation. For VAD, we instantiate an autoregressive denoising score matching (ADSM) mechanism, which progressively accumulates anomalous evidence via autoregressive denoising, enabling enhanced perception of local modes beyond visual cues. For VAA, we extend the same architecture by incorporating a lightweight auxiliary decoder and a novel self-distilled denoising score matching (SDSM) mechanism. By constructing supervision from output discrepancies instead of relying on unavailable future ground truth, our method achieves efficient training suitable or early anomaly anticipation. Extensive experiments on multiple benchmark datasets demonstrate state-of-the-art performance in both VAD and VAA while maintaining high efficiency, establishing a unified and scalable pipeline from anomaly detection to anticipation.

0.11

NOVA: Normal-Side Modeling for Training-Free Zero-Shot Video Anomaly Detection

cs.CV ·2026-09-06 ·Wei-Chih Yin, Yun-Ching Kao, Cheng-Kuan Lin, Yu-Chee Tseng
similarity 0.70
Abstract

Training-free zero-shot video anomaly detection (ZS-VAD) leverages vision-language models (VLMs) to localize anomaly instances from a predefined anomaly vocabulary, without providing any video. Existing CLIP-based methods often emphasize anomaly-side semantics, while the competing normality side remains less carefully formulated. We identify two key limitations in existing solutions: (i) blurred decision boundary: normal prompts may contain ambiguous verbs, such as running, that are semantically close to anomalies, reducing normal and abnormal separation in the VLM embedding space; and (ii) modality gap: poor alignment between features of textual normal anchors and visual frames. We propose NOVA, a training-free ZS-VAD framework that strengthens the normal side at both linguistic and visual levels. NOVA introduces Normality-Aware Prompt Construction (NA), which excludes anomaly-adjacent verbs and biases normal descriptions toward static, low-motion scenes. To overcome the text-vision modality gap, NOVA constructs a Visual Normality Anchor (VNA), which creates a weighted visual normal anchor from the initial frames of each test video, providing a video-specific normal reference without task-specific training or annotations. NOVA achieves 89.86 percent AUC on UCF-Crime and 95.07 percent AUC and 84.82 percent AP on XD-Violence, reaching state-of-the-art performance among comparable training-free zero-shot methods.

0.17

Agentic Ensemble with Vision-Language Models for Visual Anomaly Detection

cs.AI ·2026-10-20 ·Shangquan Sun, Wenjun Luo, Xu He, Yoonsoo Lee, Shijian Lu
similarity 0.69
Abstract · 10 pages, 2 figures, 13 tables

Visual anomaly detection aims to detect abnormal objects, while anomaly samples are often scarce during model training. Assembling multiple pretrained detectors can help by leveraging their transferable image scores and anomaly maps. However, most existing ensemble methods apply fixed fusion rules over saved predictions which struggle when pretrained detectors clearly disagree due to different training objectives, priors, and visual representations. To this end, we design AnomalyAgent, a post-training fusion framework for accurate visual anomaly detection. AnomalyAgent aligns heterogeneous scores and maps, summarizes score- and region-level disagreement, and treats fusion as a finite action-selection problem. It implements two types of actions: 1) score actions that determine how to fuse image-level detector confidence via consensus, robust aggregation, detector weighting, or score-map consistency; 2) map actions that fuse dense anomaly responses via consensus, detector weighting, geometric aggregation, or disagreement correction. A constrained vision-language model (VLM) is introduced to inspect high-conflict samples and select valid score action and map action to produce the final prediction. We evaluate AnomalyAgent on MVTec AD and VisA as primary benchmarks, together with seven additional public anomaly benchmarks and Real-IAD for large-scale score-level evaluation. Across these benchmarks, AnomalyAgent consistently improves over equal-weight fusion and achieves the best mean image AUROC and pixel AP among the compared fusion methods on MVTec AD and VisA, while demonstrating robust transfer across additional anomaly domains.

0.21

How to Explain your Unexplainable Video Anomaly Detector

cs.CV ·2026-11-16 ·Giacomom D'Amicantonio, Egor Bondarev
published similarity 0.68
Abstract · This paper has been accepted at the RWS workshop WACV 2027

Weakly-supervised Video Anomaly Detection (VAD) models assign anomaly scores from high-dimensional features, and their predictions remain unexplained. Saliency maps show where a model looks but not what it relies on, and the descriptions produced by Vision-Language Models (VLMs) are generated by branches that the anomaly score never reads. To address this faithfulness gap, we propose \textbf{Explain The Unexplainable (ETU)}, a classifier-agnostic framework that audits frozen VAD models through counterfactual interventions in their feature space. ETU maps video features to a discrete dependency graph of agents, actions and outcomes ($A_r \to A_a \to A_o$), regenerates the features by graph-conditioned diffusion with selected nodes of the graph masked, and measures the change in the score of a frozen detector; one set of interventions is applied unchanged to every detector. The generator is trained only to reconstruct the input features, so it cannot learn the contrast it is asked to measure, and we validate the audit with experiments that test every alternative explanation of its results, from the cost of regeneration to random deletions of matched size. On UCF-Crime features obtained from a CLIP encoder, we show that substituting the scene semantics results in the largest drop for most detectors, and that removing the outcome chain of the graph ($Δ_{rel}$) changes the anomaly score of all six audited detectors far more than removing the same number of random nodes, even though the random deletion perturbs the features about as much. We show that this targeted behavior comes from the divergence objective of the graph embedding. Without it, the two interventions become indistinguishable, and with a divergence objective in the generator as well, the score change of the detector follows the displacement of the reconstruction rather than the semantics that were removed.

0.33

AnomalyCraft-700K: Component-Level Controllable and Verifiable Synthetic Anomalies for Fine-Grained Video Anomaly Understanding

cs.CV ·2026-09-07 ·Yuzhou Long, Haodong Zhang, Yunpeng Yang, Peng Wu, Guansong Pang
similarity 0.67
Abstract · 9 pages,2 figures

Progress in video anomaly understanding (VAU) has long been limited by inherent deficiencies of real-world anomaly videos, which are hard to collect and offer little control over their content. Synthetic anomaly approaches partially alleviate data scarcity, yet their generation remains largely controlled at the category or prompt level. They also lack component-level verification of video-text consistency and provide insufficient hard normal samples near the normal-anomaly boundary. To address this, we present AnomalyCraft-700K, a component-controllable synthetic anomaly dataset for fine-grained VAU, containing over 40K videos and over 700K task-level textual annotations. From fine-grained semantic components and a progressive three-stage pipeline, we craft anomaly events that are richly detailed, semantically controlled, and temporally structured, and additionally construct per-category hard normal samples to prompt the model to discriminate based on anomaly semantics rather than surface visual cues. Moreover, using the components as verification units, AnomalyCraft-700K further performs component-wise correction of video-text discrepancies introduced during generation, providing reliable annotations with verified cross-modal alignment for six tasks that progress from anomaly detection, through anomaly retrieval and captioning, to fine-grained anomaly reasoning. Evaluations of widely used methods under both traditional and MLLM-based protocols demonstrate that AnomalyCraft-700K serves as an effective source of supervision, from anomaly detection to fine-grained anomaly understanding.

0.09