CS Paper Radar

EVISTA: Unified Multimodal Video Search with Evidence-Aware Temporal Path Optimization

Minh-Huy Tran, Van-Thinh Vo, Minh-Khoi Nguyen, Duy-Tan Nguyen, Anh-Quan Nguyen-Tran, Loi Nguyen Khanh

cs.IR ·submitted 2026-10-10 03:18 UTC · v1

Abstract

In the era of big data, precise event retrieval from large video collections has become increasingly important and challenging. In particular, locating multiple events from the same video in their temporal order is difficult when different events are described by different types of evidence, such as visual content, on-screen text, and spoken words. We develop EVISTA, a unified multimodal video retrieval system for the Ho Chi Minh City AI Challenge 2026, which integrates visual-semantic, OCR, ASR, object, and metadata search over a shared keyframe index, and propose TEMPO-P, an evidence-aware temporal path optimization method for ordered multi-event retrieval. Building EVISTA for this task reveals two limitations of independent event retrieval: a relevant video may be missed when one event is weakly retrieved, and independently retrieved event candidates may be temporally inconsistent or unable to represent events defined primarily by text or speech. TEMPO-P addresses these limitations by aggregating candidate videos across events and modalities, jointly scoring keyframes using visual, OCR, and ASR evidence, and selecting a compact temporally ordered path within each video through dynamic programming. On five organizer queries and 36 additional multi-event queries, TEMPO-P matches visual-only temporal alignment on visually described events (R@1 0.42 for both) and retrieves queries containing text-defined events, which visual-only alignment cannot represent (R@1 0.38). TEMPO-P is integrated into EVISTA as its ordered-event retrieval mode.

Comment
15 pages, 6 figures

Closest papers

Nearest neighbours by meaning across the whole index.

MultiVENT-Raw: A Benchmark for Retrieval and Reasoning over Raw Videos

cs.CV cs.IR ·2026-09-23 ·Reno Kriz, David Etter, Alexander Martin, Cameron Carpenter, Debashish Chakraborty +9 more
benchmark similarity 0.76
Abstract

Online information is increasingly consumed in video format. Much of this comes in the form of *raw video*: continuous footage taken on a cell phone, with a hand-held camera, or via CCTV, which is then directly uploaded to social media platforms and content sharing services. Whereas professional or even amateur-edited footage tends to feature scripted speech, chyrons, graphics, and metadata that help contextualize its subject matter, raw video typically contains none of these things, making it a much more challenging medium for information retrieval and machine understanding. To facilitate progress in this domain, we release MultiVENT-Raw, a multilingual collection of nearly 120,000 primarily raw videos (over 5,300 total hours), paired with 130 events and 222 event-centric queries, along with human-annotated video relevance judgments and human-extracted key facts for relevant videos. MultiVENT-Raw supports both a retrieval task---to identify videos in the collection relevant to a query event---and a generation task---to summarize event-related videos into a coherent report for a target user. We benchmark strong baselines on MultiVENT-Raw, showing both tasks to be challenging even for some of the latest multimodal models.

0.07

MELON: A Large-Scale Dataset for Multi-Event Text-to-Long-Video Retrieval

cs.IR ·2026-08-31 ·Chan Hur, SeungWoo Song, Jeong-hun Hong, Won Jun Oh, Hyeyoung Park +1 more
dataset similarity 0.74
Abstract

Existing text-video retrieval datasets primarily consist of short-form clips containing a single dominant event. While suitable for measuring basic vision-language alignment, they are limited in capturing real-world retrieval scenarios, where long-form videos naturally contain multiple semantically distinct events and a single text query may correspond to several non-contiguous temporal segments. To bridge this gap, we introduce MELON, the first large-scale dataset designed to extend text-video retrieval to long-form videos featuring complex, multi-event structures. MELON explicitly annotates multiple event intervals per video along with their corresponding textual descriptions, enabling both training and evaluation of multi-event understanding in long, untrimmed videos. In addition, we propose a multi-event aware loss that encourages models to differentiate between full-event and partial-event matches, yielding substantial improvements in retrieval accuracy. Together, the MELON dataset and our proposed loss establish a robust foundation for expanding text-to-video retrieval to complex long-form scenarios and provide a more realistic evaluation setting for future research in the field.

0.16

EMMA: Event-Driven Multimodal Memory System with Multi-Agent Retrieval

cs.CV cs.AI ·2026-09-28 ·Ruochen Zhao, Jiazheng Kang, Gezi Li, Dongying Lin, Ting Bai
top venuecode/data similarity 0.74
Abstract

Long video understanding requires reasoning over hours of content that far exceed the context window of multimodal large language models (MLLMs). Existing approaches compress video into fixed-length clips, yet events are fragmented and unrelated content is merged, yielding noisy retrieval. To overcome this challenge, we propose EMMA, an Event-driven Multimodal Memory system with Multi-Agent retrieval, which organizes memory around semantic events in three stages: (1) Event segmentation: a dual-stream detector adaptively locates event boundaries by fusing visual semantic drift with spoken transitions, rather than fixed windows. (2) Memory construction: built on the detected events, a three-level hierarchy of raw video, clip-level memory, and event-level memory stores content at progressively higher semantic abstraction. (3) Memory retrieval: a multi-agent framework collaboratively queries this hierarchy through query decomposition, tiered search, evidence validation, and answer generation, narrowing evidence from coarse events to precise clips. Extensive experiments on three challenging benchmarks demonstrate substantial improvements, showing the effectiveness of our event-aligned memory system. Code, prompt templates, and reproduction scripts are available at https://github.com/BAI-LAB/EMMA.

0.46

EC-RAG: Event Chain Retrieval-Augmented Generation for Long Video Understanding

cs.CV ·2026-10-06 ·Yuhao Qin, Junbo Wang, Yuke Li, Yining Zhu
similarity 0.73
Abstract · 12 pages, 7 figures, 7 tables, including supplementary material

Current large video-language models (LVLMs) still face challenges when dealing with long videos, mainly because frames are often processed independently, making it difficult to capture temporal dependencies across events. Although retrieval-augmented approaches have been introduced to provide additional context, most of them operate at the frame or snippet level, which limits their ability to model how events evolve over time and relate to each other. In this paper, we propose Event Chain Retrieval-Augmented Generation (EC-RAG), a training-free framework that organizes video content into an explicit event chain before question answering. Instead of retrieving isolated frames or text segments, EC-RAG first partitions the video into semantically coherent segments, represents each segment using multi-modal signals, and then links them into a structured chain that preserves temporal order and captures inter-event relationships. Given a query, the system identifies relevant events within this chain and gathers supporting evidence from the associated modalities. Our approach offers several practical advantages: (i) event-level abstraction that better reflects how video content is naturally structured, enabling more reliable localization compared to frame-level retrieval; (ii) structured multi-modal fusion that aggregates speech, text, and visual cues at the event level, allowing complementary information to be more effectively utilized during reasoning; and (iii) plug-and-play compatibility with existing LVLM backbones, requiring no additional training or reliance on proprietary models. Experiments on Video-MME, MLVU, and LongVideoBench show that this event-centric design consistently outperforms frame-level retrieval baselines, highlighting the importance of modeling temporal structure for long-video understanding.

0.16

VTG-RAG: Long-Video Reasoning via Query-Adaptive Coarse-to-Fine Sampling and Temporal Graph Retrieval

cs.CV ·2026-11-11 ·Yao Lu, Yunhao Wang, Yaxin Gao, Feifei Teng, Chenxiao Liu +5 more
similarity 0.69
Abstract

Long-video reasoning requires a system to localize sparse evidence over extended temporal sequences, preserve temporal structure, and generate answers despite substantial multimodal distractors. Existing video retrieval-augmented generation methods typically build indexes from uniformly sampled video segments. In long-video settings, this design readily introduces irrelevant context and is unreliable for questions with explicit temporal constraints. We propose VTG-RAG, a framework for long-video question answering that combines query-adaptive coarse-to-fine sampling with temporal graph retrieval. First, a two-stage sampling strategy selects core segments with high mean query relevance or high intra-segment relevance variance. The framework then integrates subtitles, speech transcripts, visual descriptions, entities, and relations from these segments into a temporally ordered multimodal graph. Finally, it performs temporally filtered, two-pass retrieval and reasoning across frame, text, and graph evidence spaces. A lot of experiments have proven the effectiveness of the VTG-RAG. Specifically, we evaluate the framework on Video-MME and LongVideoBench. On LongVideoBench, the reported Qwen2.5-VL-32B and 72B results improve from 48.16% to 50.56% and from 52.13% to 53.32%, respectively, with 24 answer-input frames. These correspond to gains of 2.40 and 1.19 percentage points. The ablation experiments of VTG-RAG further support that the contributions of each component are very important.

0.05

Structured Spatio-Temporal Evidence Graphs for Open-Vocabulary Object Retrieval in Videos

cs.DB ·2026-09-20 ·Jingdan Wang, Chuanwen Li
similarity 0.69
Abstract

Open-vocabulary object retrieval in videos requires answering free-form object queries under bounded query-time cost. Existing index-based systems typically store independent frame-level regions and retrieve them with vision-language similarity, which is effective for appearance queries but mismatched with predicates whose evidence is temporal or relational, such as stopped state, scene-region occupancy, persistence, and object interactions. We identify this gap as an evidence-unit mismatch: the query is expressed over tracklets or object tuples, while the index stores isolated boxes. To address it, we propose STEG-OVR, a structured spatio-temporal evidence graph for open-vocabulary object retrieval. STEG-OVR represents persistent objects as tracklet nodes and temporally compatible object pairs as relation edges, storing appearance, motion, scene occupancy, relative geometry, velocity compatibility, and symbolic relation evidence. A query is decomposed into entity, state, scene, temporal, and relation slots, which activate only the corresponding retrieval channels before soft score fusion and fixed-budget consistency verification. Diagnostic experiments on three object-centric settings show AP improvements from 0.0882 to 0.2073 on Beach, from 0.0834 to 0.1505 on Shibuya, and from 0.701 to 0.743 in a LOVO-style comparison. The gains are strongest for stopped-state and scene-region-occupancy queries, while sustained relations remain sensitive to tracking continuity and predicate calibration.

0.14

EM^2Mem: Event-Centric Multimodal Memory for Large Language Models

cs.CL cs.AIcs.LGcs.MM ·2026-09-01 ·Yijun Chen, Yaqi Zheng, Yanya Li, Boyi Xiao, Buqiang Xu +9 more
top venuecode/data similarity 0.69
Abstract · Accepted by EMNLP 2026 findings

Multimodal memory offers a scalable interface for long-video question answering, but existing methods often retrieve captions, frames, transcripts, summaries, or graph facts as isolated fragments. Although searchable, such fragments are not generation-ready: language models must reconstruct cross-modal and temporal alignments at inference time, when context is limited and attribution is difficult. We propose EM^2Mem, an event-centric multimodal memory framework that binds heterogeneous evidence to event anchors during memory construction. Each event-indexed memory cell aligns multimodal records, temporal context, graph-linked relations, semantic facts, and provenance, enabling compact evidence readout over grounded multimodal events rather than modality-specific fragments. Across three long-video QA benchmarks, EM^2Mem improves average accuracy over the strongest memory baseline by 2.0, 2.4, and 3.7 points, improves strict event-level Top-5 evidence recall by 7.0 points, and reduces per-query latency by 4.67 times and total inference tokens by 63.66% (The code will be integrated into https://github.com/zjunlp/LightMem).

0.66

TF-PRVR: Training-Free Partially Relevant Video Retrieval

cs.CV ·2026-10-06 ·Giyeol Kim, Chanho Eom
similarity 0.68
Abstract

Partially Relevant Video Retrieval (PRVR) aims to retrieve untrimmed videos containing moments relevant to a given text query. Despite recent progress, existing PRVR methods suffer from two key limitations: a fixed video decomposition scheme that causes semantic dilution, and source-domain overfitting induced by task-specific training. In this paper, we propose TF-PRVR, the first training-free framework for PRVR. TF-PRVR leverages frozen vision-language features to construct video-specific hierarchical representations. It derives temporal semantic signals from frame-level features and applies frequency-based multi-scale analysis to identify adaptive temporal boundaries, producing hierarchical segments with coherent event-level semantics. Built on these segments, TF-PRVR constructs a unified multi-scale graph and propagates query relevance across temporally and semantically related nodes. A moment-aware scoring strategy then aggregates temporally aligned relevance across scales, emphasizing consistently supported moments while suppressing isolated false responses. Without task-specific training, TF-PRVR preserves the general-purpose alignment capability of pre-trained vision-language models and avoids dataset-specific overfitting. Extensive experiments demonstrate consistent performance across datasets with diverse visual and temporal characteristics, suggesting a practical direction for training-free PRVR.

0.13