CS Paper Radar

OneReel: Efficient Multi-Scene Video Generation with Scene-Structured Attention

Wei Zhang, Yuchen Wang, Tianrui Feng, Hongrui Wu, Chenfeng Xu, Fan Lai

cs.DC ·submitted 2026-11-07 09:01 UTC · v1

Abstract

Multi-scene video generation must follow scene-specific prompts while preserving consistency across scenes. Dense joint generation enables cross-scene information exchange but incurs quadratic attention cost, whereas separate generation avoids this cost at the expense of cross-scene consistency. We present OneReel, a training-free system that exploits scene structure to enable efficient multi-scene generation. Our key insight is that multi-scene generation can preserve complete intra-scene attention while restricting cross-scene information exchange to a bounded shared memory. OneReel uses scene prompts and durations to establish scene-specific attention scopes and dynamically selects a bounded set of representative latent frames for cross-scene sharing. This reduces video attention complexity from quadratic to linear in the number of scenes. To translate these savings into execution speedups, OneReel packs scene-local and shared K/V into dense variable-length attention and employs work-balanced multi-GPU partitioning with communication-aware selection between K/V gathering and Ulysses-style parallelism. Our evaluations across four video generation models show that OneReel achieves 1.7-4.9x end-to-end speedup over dense attention and 1.2-2.2x over existing sparse-attention advances, while improving text-following accuracy by 34-37 percentage points and maintaining cross-scene consistency.

Closest papers

Nearest neighbours by meaning across the whole index.

MosaiChunk: Compositing Spatio-Temporal Memory for Autoregressive Video Generation

cs.CV cs.GR ·2026-10-01 ·Yiwen Zhang, Haocheng Xi, Michael Tian-Yue Liu, Alexei A. Efros, Hadar Averbuch-Elor +2 more
benchmarkcode/data similarity 0.69
Abstract · 27 pages. Project page: https://mosaichunk.github.io/

Long-horizon autoregressive video generation is limited by a finite context window. When an object or scene falls out of context, its fine-grained visual details may be lost and difficult to recover upon reappearance. To retain access to such visual details, we introduce MosaiChunk, a spatio-temporal memory mechanism that composes a mosaic of selected historical key-value (KV) entries across space and time. Our approach is motivated by the observation that a frozen video generator can directly consume such non-contiguous historical KV and recover the corresponding visual content. We therefore keep the generator fixed and learn only a lightweight router that determines which historical sections to include in the mosaic under a fixed active-memory budget. We further introduce RememBench, a benchmark of long-horizon revisits with prompt-driven text-to-video (T2V) and camera-driven image-to-video (I2V) splits. Our experiments show that MosaiChunk consistently improves revisit consistency over both sliding-window inference and whole-chunk retrieval under matched memory budgets, across both T2V and I2V settings.

0.54

MixPT: Point Transformer with Interleaved Window and Global Attention

cs.CV ·2026-10-22 ·Minkyun Seo, Hyungtae Lim, Jaesik Park
code/data similarity 0.68
Abstract · 23 pages, 14 figures, 12 tables. Project page: https://minkyunseo.github.io/MixPT/

Point cloud transformers aim to capture detailed local geometry and relationships across a scene at a manageable computational cost. Two common approaches are window attention and global attention. They differ in how they enable scene-wide interaction, which refers to information exchange among tokens across a scene. Window attention reduces computational cost by restricting self-attention to groups of tokens called windows; in serialized designs, each window contains a fixed number of consecutive tokens from a spatially ordered sequence. Because each block only directly connects tokens within the same window, information exchange across windows relies on changes in window membership across blocks and hierarchical pooling. Global attention, in contrast, directly connects all tokens within a scene, but computing attention over all token pairs can be costly when the number of tokens is large. To balance computational cost and scene-wide interaction, we introduce MixPT, a point transformer that uses a mix of window and global attention blocks in an interleaved schedule, while retaining a single fixed serialization pattern for its window blocks. For scenes with a large number of tokens, MixPT applies global attention to spatially pooled tokens and returns residual updates to the original-resolution stream. Across indoor and outdoor benchmarks, MixPT achieves competitive accuracy with lower latency than representative window and global attention baselines. We further demonstrate the feasibility of scaling the attention stage in depth by repeating the interleaved pattern of window and global attention blocks.

0.29

Eco3R: Pinned-Key Inference for Efficient and Streamable Feed-Forward 3D Reconstruction from Long Videos

cs.CV cs.AI ·2026-11-13 ·Hongsen Cao, Shanxin Yuan, Mona Jaber, Ahmed Sayed
similarity 0.68
Abstract · 26 pages, 10 figures. v2: renamed the method from EcoVGGT to Eco3R, added Figure 1, and revised the text

Feed-forward multi-view transformers such as VGGT recover the cameras and geometry of a video in one pass, but the cost of their cross-frame self-attention grows quadratically with the frame count, and their accuracy drops on long videos. Most remedies approximate attention over all frames under a fixed computational budget or select a few keys by their content. We hypothesize that the keys of a long video can be selected independently of video content, and our experiments on VGGT and $π^3$ support it: a small number of uniformly sampled frames can serve as the keys of every frame. We present Eco3R, a training-free inference framework whose keys form a Pinned Key Set of 31 such frames. A pin pass stores their tokens in a Scene Cache, and the other frames read it in independent batches that reproduce joint inference with these keys exactly, at a constant cost per frame. An Online Stream updates the pinned frames during recording. Six middle global blocks align the frames and form a reusable VGGT module: the Alignment Score reads their attention to flag failures, and Alignment Refocusing reduces them by fine-tuning only these blocks (3.6 GPU hours). Eco3R ($π^3$) matches $π^3$'s success rate on 1,000-frame ScanNet videos while running 13.8 times faster. With refocusing, Eco3R (VGGT) raises VGGT's success rate from 70% to 82% while running 8.5 times faster on 500 frames. Eco3R can reconstruct sequences of up to 4,000 frames on a single A100 GPU, while Eco3R ($π^3$) streams 1,000 frames at 18.6 fps, achieving the lowest online absolute trajectory error.

0.13

CaT-Movie: Balancing Global Planning and Local Rendering for Long Multi-Shot Video Generation

cs.CV ·2026-11-19 ·Jinpeng Lu, Haoyuan Shi, Yuanshen Guan, Jiyuan Xia, Bin Wu +2 more
benchmarkdatasetcode/data similarity 0.68
Abstract · 31 pages

Current multi-shot video generators render only limited durations per pass, requiring long movies to be stitched from local segments across which narrative progression and visual identity drift. To synthesize long-horizon movies with Coherence across Time, we present CaT-Movie, a multi-shot generation system consisting of a global story planner and a local window renderer. The planner expands user requests via a large language model to preserve, infer, and complete story details, while deterministic code maintains the resulting storyboard across shots to compile self-contained shot prompts and index reference frames from generated history. To execute this guidance, the renderer applies reference-routed sparse self-attention and shot-wise cross-attention to route identities from the indexed frames while isolating per-shot textual conditions. Given the scarcity of high-quality open-source multi-shot data, we construct a dataset of over 350K film shots annotated with semantic state transitions, entity grounding, and multi-granularity requests. We further build CaT-MBench, a benchmark on held-out films with the same annotations. It gives every baseline its native format and counts cross-shot consistency only when a cut opens a new camera setup, preventing repeated framing from passing as consistency. On CaT-MBench, CaT-Movie attains the highest average score and the most consistent characters among all methods. The project page is available at https://jinplu.github.io/CaT-Movie/.

0.32

GAD: Geometry Attention Distillation for Consistent Video Generation

cs.CV ·2026-10-13 ·Junfeng Ni, Ziyu Zhu, Fan Jiang, Yu Liu, Chiyu Wang +5 more
positioncode/data similarity 0.68
Abstract · Project page: https://fantasy-amap.github.io/GAD/

Video diffusion models generate visually compelling short-term sequences but often struggle to preserve scene geometry and object structure over time. In this paper, we first demonstrate that current video attention cannot identify reliable correspondences across frames, with responses dispersed across multiple regions even after recognizable scene structure has emerged. We argue that reliable cross-frame associations are essential for information exchange as scene structure forms in video generation. Thus, we introduce Geometry Attention Distillation (GAD), which transfers correspondences from a frozen geometry model's attention to video attention. Extracted from clean videos, these correspondences link the same static surface or moving object across frames. Since structure formation begins in deeper layers at higher noise and shallower layers at lower noise, we apply attention distillation at a student layer selected according to the noise timestep. After training, video attention focuses more consistently on corresponding locations. Experiments across all three video backbones demonstrate improved long-range scene consistency and dynamic temporal consistency, with geometric gains across generation lengths. On Wan2.2 5B, GAD improves cross-frame matching accuracy by at least 30% over the Wan baseline on both long-range benchmarks. Notably, GAD achieves these gains without additional inference modules, offering a promising approach to efficient and consistent video-based world simulation. Project page: https://fantasy-amap.github.io/GAD/.

0.26

Rethinking Long-Video Efficiency: A Joint Allocation Perspective on Frames, Pixels, and Front-End Latency

cs.CV ·2026-10-03 ·Sixun Dong, Wei Li, Andong Deng, Qi Qian, Victor Zhu +2 more
top venuecode/data similarity 0.67
Abstract · Accepted at NeurIPS 2026. Project page: https://sixundong.com/projects/lohi

Efficient long-video understanding with vision-language models (VLMs) is often framed as selecting informative frames or visual tokens at a fixed native resolution. We show that per-frame resolution can instead be traded for denser temporal coverage, while front-end decoding latency depends on the size of the candidate pool rather than the final token budget. An empirical study across multiple VLMs and long-video benchmarks yields three findings: dense low-resolution sampling outperforms sparse native-resolution sampling at matched token budgets; resolution-sensitive tasks benefit from selected high-resolution frames; and front-end decoding dominates wall time for hour-long videos. Motivated by these findings, we introduce LoHi, a training-free, single-pass framework that combines a dense low-resolution video stream with sparse high-resolution image frames through the VLM's native video and image pathways. LoHi-Anchor selects high-resolution frames using codec I-frame metadata, while LoHi-SemDiv uses query relevance and visual diversity over CLIP features. Across three long-video benchmarks, LoHi improves average accuracy by 10.6 percentage points over the native-resolution baseline at a matched token budget and by 5.2 percentage points over the strongest prior efficiency method. It also reduces front-end decoding latency by up to 7x on hour-long videos. Project page: https://sixundong.com/projects/lohi

0.67

UniMem: Unifying Visual and Textual Memory in VLAs for Non-Markovian Tasks

cs.RO cs.CV ·2026-08-24 ·Lars Osterberg, Maggie Wang, Mac Schwager
similarity 0.67
Abstract

While Vision-Language-Action (VLA) models have leveraged internet-scale pretraining and task-specific finetuning to achieve strong performance on long-horizon tasks, they often struggle with non-Markovian tasks that require memory. Existing approaches to memory typically involve additional Vision-Language-Models (VLMs) for long-term memory management, introducing a memory bottleneck and a fractured training pipeline. Conditioning on multiple historical frames can provide the VLA with access to more descriptive features of past scenes, but can degrade performance if frames are chosen at arbitrary, fixed intervals. To address these limitations, we present UniMem, a framework that unifies high-level visual and textual memory and low-level control under one backbone. UniMem employs an event classifier to initiate textual and visual memory updates, a keyframe encoder for dense visual memory, and a keyframe caching technique to minimize overhead during policy rollouts. We evaluate UniMem across five simulation and four hardware tasks targeting sequential and spatial memory, demonstrating that our unified, single-model system outperforms fixed-interval image sampling baselines (93.4% vs. 68.2%) in simulation and hierarchical VLM-VLA baselines (80.0% vs. 43.5%) in hardware, while offering faster inference and a simple training pipeline for easy adoption. Project website: https://losterberg3.github.io/unimem-vla/

0.50

PixelUMM: Encoder-Free Unified Image and Video Understanding and Generation

cs.CV ·2026-09-29 ·Cong Wei, Xuanchi Ren, Bryan Chu, Weiming Ren, Huan Ling +6 more
code/datahighly cited similarity 0.67
Abstract · 31 pages, 21 figures. Project page: https://nv-tlabs.github.io/PixelUMM/

Unified Multimodal Models (UMMs) often rely on separate visual representations for understanding and generation, increasing visual context length and complicating integration with established vision-language pretraining pipelines. Recent advances in pixel-space modeling offer an encoder-free alternative, but extending this paradigm from images to videos is non-trivial: video understanding and generation adopt different temporal representations, leaving the design of a unified visual interface an open question. We present PixelUMM, an encoder-free model for unified image and video understanding and generation directly in pixel space. PixelUMM represents images as spatial patches and videos as spatiotemporal tubelets, connecting raw pixels to a shared multimodal backbone through single-layer linear projections. Its Mixture-of-Transformers architecture combines shared attention with task-specific parameters and extends clean-pixel prediction to video generation, jointly supporting autoregressive text prediction and pixel-space flow matching. Experiments show that PixelUMM achieves competitive performance across image and video understanding and generation tasks. We further conduct empirical studies of key design choices, including decoder design and spatial-temporal patch size, providing insights for future pixel-space unified multimodal models.

0.58