CS Paper Radar

Organizing Temporal Evidence in Hyperbolic Space for Video Moment Retrieval

Mengnan Liu, Sanping Zhou, Kun Xia, Jiaxuan Lin, Gang Hua, Le Wang

cs.CV ·submitted 2026-10-23 08:15 UTC · v1

Abstract

Video moment retrieval aims to localize a query-relevant temporal segment in an untrimmed video. Existing methods typically perform video-text alignment in Euclidean space, where representations are embedded in a flat geometry. However, accurate moment localization requires organizing query-relevant evidence across multiple grounding states, where target regions, boundary transitions, and contextual distractors exhibit distinct temporal structures and relevance patterns. Such Euclidean representations lack an explicit geometry to jointly capture these relationships, motivating us to explore structured temporal evidence organization from a geometric perspective. Specifically, we propose Hyperbolic Temporal Evidence Organization (HTEO), a geometric framework that organizes query-aware temporal memory in hyperbolic space. HTEO maps temporal representations into a Poincaré ball and organizes temporal evidence through Radial Evidence Organization (REO) and Orthogonal Angular Refinement (OAR). REO adjusts temporal tokens' radial positions to encode the structural differences among temporal evidence, while OAR enhances query-conditioned semantic alignment through orthogonal rotation. The organized representation is then projected back to the Euclidean tangent space and injected into the decoder through a lightweight residual memory update, enabling hyperbolic geometric cues to be incorporated into standard Transformer decoders while preserving the original cross-modal memory. Furthermore, we introduce radial structure regularization and angular relevance supervision to explicitly guide temporal evidence organization and query-conditioned semantic alignment, respectively. Extensive experiments and analyses across multiple benchmarks demonstrate that HTEO improves temporal localization while providing an interpretable geometric organization of temporal evidence.

Closest papers

Nearest neighbours by meaning across the whole index.

VideoTapestry: Query-Adaptive Memory Refinement for Multi-Agent Long-Video Understanding

cs.CV cs.AI ·2026-10-05 ·Yucheng Liu, Yufei Yin, Mingxiao Feng, Jiajun Deng, Wengang Zhou +1 more
similarity 0.65
Abstract

Long-video understanding places substantial demands on memory, as answering questions often requires retrieving information distributed across extended temporal spans. Existing approaches broadly follow two paradigms: query-driven exploration, which is sensitive to localization errors, and query-independent memory construction, which may omit question-specific details. We introduce VideoTapestry, a training-free multi-agent framework that adapts a preconstructed hierarchical video memory through coarse-to-fine, query-driven refinement. The preconstructed memory organizes video content into three levels, capturing global narrative context, event-level temporal structure, and fine-grained relational evidence, respectively. To support coarse-to-fine localization and observation, we assign a specialized agent to each level, keeping retrieval and refinement within a scale-specific context. Guided by the query, these agents revisit relevant video regions and enrich layer-wise memories with targeted multimodal observations. Their refinements are assembled according to the original hierarchy into a composite query-adaptive memory, preserving global context in a compact form while retaining fine-grained evidence along query-relevant branches for final reasoning. Compared with direct GPT-5.5 inference, VideoTapestry achieves absolute accuracy gains of 17.2%, 14.9%, 9.8%, and 7.0% on LVBench, LongVideoBench (Long), Video-MME (Long), and EgoSchema, respectively, achieving the state-of-the-art results among all competitors.

0.14

CurvSpec: Adaptive Multi-Curvature Learning for Partial Relevant Video Retrieval

cs.CV ·2026-09-29 ·Zhen Liu, Letian Li, Jinpeng Wang, Shuzhao Xie, Yuzhi Huang +2 more
published similarity 0.64
Abstract · 10 pages. Accepted to ACM Multimedia 2026 (MM '26)

Partially Relevant Video Retrieval (PRVR) seeks to retrieve untrim-med videos containing a moment that matches a text query, without temporal annotations. The relevant moment may last only seconds within a video spanning several minutes, creating an extremely low signal-to-noise ratio that makes PRVR more challenging than standard full-video retrieval. This task presents two intertwined challenges: (1) signal dilution, where coarse global representations blur the brief relevant signal into the dominant irrelevant surroundings;(2) curvature rigidity, where embedding all videos in the same fixed-geometry space distorts representations for videos that range from flat atomic events to deep compositional hierarchies. Existing PRVR methods have improved moment selection and cross-modal matching, but they still typically encode all videos in a single fixed-curvature retrieval space, limiting their ability to model diverse video structures. To address both challenges, we propose CurvSpec, a framework that learns content-adaptive curvature for video retrieval representations rather than imposing a fixed geometric prior. CurvSpec processes features through parallel Euclidean and hyperbolic attention layers, with independently learned curvatures assigned to the hyperbolic layers, and a content-aware fusion mechanism routes each input to its most suitable geometric regime. To further suppress signal dilution, CurvSpec represents each video with semantic centroids whose number is determined by the video's content complexity, projects them onto the learned manifold, and matches each query against its nearest centroid by geodesic distance. Experiments on ActivityNet Captions, TVR, and Charades-STA demonstrate state-of-the-art retrieval performance.

0.36

Temporal Tree of Thought: Reasoning-Guided Visual Cue Search for Long-Video Understanding

cs.CV ·2026-08-28 ·Ziling Huang, Shin'ichi Satoh
top venue similarity 0.64
Abstract · Accept by EMNLP2026

Long-video understanding remains challenging for Multimodal Large Language Models (MLLMs) due to limited context length. Uniform sampling may miss crucial moments, while agent-based frame video understanding methods often evaluate frames independently, overlooking the temporal organization of videos. Ideally, evidence selection should mimic how humans answer questions about long videos: first locating the relevant segment from the global context, then zooming into local objects and details. We propose Temporal Tree of Thought T^3, a training-free framework for adaptive coarse-to-fine long-video understanding. T^3 constructs a question-agnostic hierarchical temporal tree via recursive temporally constrained clustering, where each node represents a contiguous segment with an informative key frame. During inference, T^3 performs an answer-retrieve-explore loop: it reasons over coarse representative frames, generates a search statement when evidence is insufficient, and expands relevant branches for finer-grained evidence. This process adaptively shifts the search target from temporal regions to specific objects and visual details to help video understanding. Experiments on VideoMME, LongVideoBench, and LVBench show that T^3 improves Qwen2.5-VL-7B by 0.5%, 4.6%, and 4.4%, respectively, under the same frame budget, demonstrating the effectiveness of structured temporal reasoning.

0.34

Coarse Indexing, Fine Evidence: Decoupling Temporal Granularity in Long-Video RAG

cs.CV cs.AI ·2026-08-24 ·Zhe Jin, Zhimin Lin, Bin Zheng, Junhua Fang, Huihua Yang
position similarity 0.63
Abstract

Graph-based retrieval-augmented generation (RAG) provides a scalable paradigm for long-video understanding, but existing systems typically inherit a fixed temporal granularity from video segmentation when constructing their retrieval index. We argue that this design unnecessarily couples indexing granularity with evidence granularity: coarse representations can often suffice for locating relevant temporal regions, while fine-grained evidence remains important for downstream reasoning. We propose \textbf{Density-Aware Graph Construction (DAGC)}, a training-free approach that decouples a query-independent coarse retrieval index from the original fine-grained evidence space. DAGC constructs a compact, density-adaptive graph index by merging visually redundant neighboring chunks, while preserving mappings to the original temporal units. Retrieved coarse regions are subsequently expanded back to the original chunk granularity for fine-grained evidence refinement and answer generation. Experiments on MLVU, VideoMME, and LongVideoBench show that DAGC retains only about 40--50\% of the original graph nodes and achieves $1.3$--$1.7\times$ end-to-end wall-clock acceleration while preserving approximately 99\% of the original QA performance. The gains transfer across different LVLM backbones and video RAG pipelines, suggesting that long-video RAG need not maintain the same temporal granularity for indexing and evidence reasoning.

0.40

Concentrate After Imagination: Text-Conditioned Evidence Grounding for Partially Relevant Video Retrieval

cs.CV ·2026-09-08 ·Shuaiqi Cheng, Siyu You, Yanbi Wu, Yuxi Chen, Jiahao Zhang +1 more
similarity 0.62
Abstract

Partially Relevant Video Retrieval (PRVR) retrieves untrimmed videos when queries describe only short moments. Although recent methods improve local representations, uncertainty modeling, and global context, final ranking often still trusts the strongest local response; a coincidentally similar fragment can therefore produce an unsupported peak. We identify this failure as the query-agnostic concentration bottleneck and propose TRACE, a score-level evidence verification operator for PRVR. Given a query and global video registers, TRACE activates query-relevant registers, routes their support to frame-level evidence, and smoothly marginalizes alternative query-to-register-to-frame paths before localized temporal selection. Unlike representation-level feature fusion, TRACE uses this evidence only as a query-conditioned residual calibration of the original local score. On ActivityNet Captions, Charades-STA, and TVR, TRACE achieves the best SumR on all three benchmarks and improves the DreamPRVR backbone by 1.2, 1.1, and 1.5 points, respectively. Ablation, routing-corruption, hard-negative, and cross-backbone transfer analyses support the interpretation that the gains arise from query-conditioned evidence verification rather than a generic score offset.

0.18

SE-Mem: Semantic-Episodic Memory for Scene-Consistent Generative View Synthesis

cs.CV ·2026-10-31 ·Changsong Lei, Yihan Li, Yu-Hui Wen, Huankun Sheng, Ming Li +3 more
similarity 0.61
Abstract · 13 pages, 6 figures

Generative view synthesis aims to synthesize novel-view videos from reference observations along user-defined camera trajectories. Maintaining global scene consistency throughout this process remains challenging for camera-conditioned interactive video generators. Existing approaches typically select reference frames based solely on camera poses, overlooking whether the content remains semantically relevant or even visible. To address these limitations, we draw inspiration from semantic and episodic memory in human cognition, in which semantic memory captures generalized knowledge across experiences, whereas episodic memory preserves information tied to specific past events and contexts. Motivated by the complementary roles of semantic and episodic memory, we introduce SE-Mem, a Semantic-Episodic Memory framework that organizes historical observations in a structured 3D representation and augments its elements with semantic importance scores. A two-stage memory retrieval strategy is designed to identify the most informative frames to provide contextual guidance. Taking into account both semantic significance and geometric visibility, SE-Mem preserves semantic coherence and fine-grained visual details during novel view synthesis. Extensive evaluations show that our approach improves visual quality and achieves competitive geometric consistency across multiple benchmarks.

0.16

Learning to Retrieve: Internalizing Memory Retrieval for Video World Models

cs.CV ·2026-10-08 ·JiaKui Hu, Tailai Chen, Yuqi Pan, Xuerui Qiu, Jialun Liu +6 more
similarity 0.61
Abstract

Video world models aim to generate explorable, 3D-consistent scene videos conditioned on camera trajectories. Existing approaches often rely on external memory systems that explicitly retrieve previously observed content to mitigate scene drift during long-horizon generation. However, these auxiliary memory pathways operate outside the model's internal generative dynamics, preventing the model from intrinsically learning when and what historical information should be retrieved. We propose to internalize memory retrieval into the generation process, allowing retrieval to emerge as an intrinsic behavior of the video world model rather than relying on an external memory system. Based on this principle, we introduce \textbf{Learning-to-Retrieve (L2R)}, which repurposes the model's persistent internal state as a memory for historical context. A camera-conditioned retrieval gate selectively accesses relevant historical information from this state, determining \textit{what to retrieve}, while a retrieval trigger determines \textit{when to retrieve}. We further supervise the trigger with a 3D re-visibility signal, activating retrieval when previously observed content re-enters the current view while otherwise preserving the existing context. Together, these components enable the model to intrinsically acquire memory retrieval behavior and incorporate relevant historical observations into generation without a separate retrieval pathway. Across multiple base models and camera-revisit benchmarks, L2R improves long-term scene consistency while eliminating the need for an external memory bank or 3D conditions. https://jkhu29.github.io/l2r

0.45

SENRI: Understanding the Space-Event Nexus in Real-Life Videos

cs.CV ·2026-10-18 ·Kehan Lan, Kaining Ying, Henghui Ding, Kaipeng Zhang
code/data similarity 0.61
Abstract · Project Page: https://senri-video.github.io/

Human memories are organized in a way that connects events with their corresponding spatial contexts. An AI assistant should also be able to memorize and recall the nexus of events and spaces. Thus we introduce SENRI, a task that studies this space--event connection in long, continuous real-life egocentric video. It contains six task types in two directions: Event$\rightarrow$Space, which recovers spatial knowledge from event cues, and Space$\rightarrow$Event, which retrieves events from regions and trajectories. To evaluate this, we build SENRI-Bench, containing 500 manually annotated multiple-choice questions over 10 uninterrupted self-recorded videos with around 25.3 hours of content. Also, we evaluate three typical long video understanding paradigms: direct perception, evidence seeking, and persistent memory. The task remains challenging across all three paradigms, highlighting substantial room for improvement. Then we show two representative failure modes of existing memory methods and address these evidence gaps with SENRI-Agent, a controller that uses Target-Guided Visual Check (TGVC) to propose local visual check tasks for parallel verifiers and integrate the reports into a working memory to help with reasoning. Built upon existing memory systems, SENRI-Agent improves the performance of existing baselines without modifying their stored memories. We hope SENRI will provide valuable insights into how real-life egocentric understanding should be evaluated, reveal the gaps in existing approaches, and offer a possible direction for improvement.

0.26