CS Paper Radar

From Retouched Images to Paired Videos: A Synthetic Benchmark for Face Video Retouching

Peiyuan Zeng, Fengchuang Xing, Shiming Wu, Hongwei Yue, Chao Zhou

cs.CV ·submitted 2026-11-13 01:16 UTC · v1
benchmarkcode/data

Abstract

Paired unretouched/retouched face videos are costly to produce because professional editing must remain consistent over time, while releasing identifiable face footage raises privacy, consent, and governance concerns. We present DeepVFR, an initial controlled synthetic benchmark-not a new retouching architecture-for paired face video retouching. Starting from paired synthetic DeepFRR images, Wan2.2 generates mild-motion source clips, and backend-specific face transfer constructs synchronized proxy retouching targets, followed by automatic validity screening and manual visual inspection. DeepVFR contains 1,024 pairs of 150-frame clips at 30 fps, evaluated under an identity-disjoint protocol with explicit frame correspondence; release metadata will record the construction route. Within-benchmark baselines, external checkpoints applied to DeepVFR, qualitative analysis, and a 17-participant ranking study establish reference behavior in the controlled domain. The constructed targets exhibit a substantial low-preference tail in the human study. Because both streams inherit synthetic content and the targets additionally inherit commercial-API or face-transfer artifacts, the benchmark neither replaces artist-retouched captured video nor establishes synthetic-to-real generalization. DeepVFR provides a starting point for paired supervision and controlled evaluation. Data, individual study rankings, and training logs will be made publicly available upon acceptance. Broader motion coverage and real-video validation remain future work.

Closest papers

Nearest neighbours by meaning across the whole index.

From Short Clips to Real-Time Streams: Identity-Aware Supervision and Context-Efficient Generation for Video Face Swapping

cs.CV ·2026-10-19 ·Zekai Luo, Yuwei Liang, Ruiqi Ma, Haoyu Huang, Xuehao Wang +6 more
similarity 0.74
Abstract · Project webpage: https://aim-uofa.github.io/StreamingSwap/

Video face swapping transfers the reference identity to a target video while preserving its other attributes and motion. Despite recent progress in reference-guided flow-based methods, jointly achieving faithful identity transfer, high fidelity, and real-time generation over long video streams remains challenging. To address this challenge, we rethink the task-specific structure of reference-guided video face swapping. First, face swapping changes only identity while preserving most visual content, yet flow-matching supervision is weakly sensitive to these changes. We therefore develop Joint Rec-ID Loss, which balances explicit identity supervision and latent reconstruction according to the noise level. Second, the target video already provides non-edited attributes and temporal dynamics, enabling generation with fewer spatial tokens and shorter temporal contexts through Face-RoI Generation and Short-Context Rollout Training. By integrating these designs, we propose StreamingSwap, a reference-guided flow-matching framework for real-time streaming video face swapping. We also introduce HumanFaceVid, comprising 1,000 hours of cinematic training data and evaluation sets of 200 short and 200 long videos. Experiments demonstrate that our method achieves competitive identity transfer and visual quality at 20 FPS on a single NVIDIA RTX 4090 GPU, with stable generation over videos up to 10 minutes long. Project webpage: https://aim-uofa.github.io/StreamingSwap/

0.32

Not All Challenges Are Equal: Evaluating Interactive Liveness Detection Against Real-Time Face Swaps

cs.CV cs.CR ·2026-10-12 ·Anton Firc, David Drtil, Martin Perešíni, Milan Šalko, Vojtěch Staněk +1 more
publishedhot direction similarity 0.67
Abstract · Accepted at BIOSIG 2026

Remote identity verification assumes that a camera stream shows a live person, yet public real-time face-swap tools can replace that person before the stream reaches the verifier. Passive deepfake detectors generalize poorly to new generators, while interactive challenges are often selected without evidence of which gestures expose current attacks. We evaluate randomized gesture challenges in 108 live sessions with six subjects, 24 impersonated identities, and two face-swap tools, DeepFaceLive and FaceFusion, injected through a virtual camera. The static calibration segment reaches a session-level AUC of 0.878 with a frozen UCF spatial detector, while the mean across the active-gesture windows reaches 0.988. Pose and combined gestures have the highest point estimates, whereas expressions show no clear improvement. Hand occlusion against FaceFusion, configured with an occlusion mask, scores below the static baseline. The end-to-end time required to satisfy the validator also distinguishes attacks from genuine sessions, achieving a session-level AUC of 0.732, although this signal includes synthesis and transport delays. These results show that challenge sets should be tested against current attack configurations and that response time can provide an additional, deployment-specific signal.

0.61

Edit in Sync: Unified End-to-End Audio-Visual Editing for Human-Centric Videos

cs.CV ·2026-10-20 ·Ziye Yuan, Minghong Cai, Xu He, Ruchang Yao, Xiao Fu +6 more
similarity 0.67
Abstract · 13 pages, 5 figures, 4 tables

Human-centric audio-visual editing requires precise alignment of lip movements, gestures, and expressions with speech. Existing methods process inputs through independent video and audio pipelines, preventing cross-modal interaction between visual editing and audio dubbing. Many of these pipelines further rely on mask-based inpainting. As a result, they suffer from poor lip-audio synchronization, cumulative pipeline errors, and an inherent mask-size dilemma. We present AVE, the first unified end-to-end framework that jointly performs reference-guided person swapping, insertion, and deletion across both audio and video modalities without requiring spatial masks. To overcome data scarcity, we design a synthesis pipeline and introduce AVE-Dataset, the first audio-visual editing dataset to provide 100K pairs with appearance and voice references. By combining Synthetic-to-Real reconstruction with preliminary audio warm-up, our framework transcends synthetic pipeline limits and mitigates audio-video token imbalance. AVE outperforms modality-separate audio-visual editing pipelines in editing realism, temporal consistency, and audio-visual alignment.

0.17

BabelFake: A Multilingual Audio-Visual DeepFake Benchmark

cs.CV ·2026-10-05 ·Carlotta Segna, Joel Tschesche, Anna Rohrbach
benchmark similarity 0.66
Abstract

Reliable and practical audio-visual DeepFake detection requires benchmarks that reflect diverse linguistic contexts and modern data synthesis pipelines for visual as well as audio manipulations. However, existing datasets predominantly contain footage of English-speakers, often include outdated manipulation types, or overlook the audio modality. Further, many datasets feature individuals who did not consent to be used in DeepFake creation. We introduce BabelFake, a multilingual audio-visual DeepFake benchmark recorded with consenting participants. BabelFake contains 399k clips (1,323 hours) from 496 individuals spanning five languages (English, German, Italian, French, Spanish). Our modular data generation pipeline pairs 11 modern video manipulation methods with 4 voice cloning engines, distinguishing visual-only (face swapping) and joint audio-visual manipulations (lip synchronization and portrait animation). By benchmarking state-of-the-art detectors, we show that detection difficulty depends on the audio-visual generation pairing, with substantial performance degradation when authentic audio is preserved. Cross-language/demographic evaluation reveals sensitivity varying across detector architectures and training data, while human evaluation reveals that perceived realism and machine-detection difficulty do not necessarily align.

0.16

Doctor Trigger: A Framework for Release-Bound Face Verification

cs.CR ·2026-10-12 ·Chia-Ming Lee, Chia-Yu Lin, Hung-Kai Huang, Yi-Ting Ku, Yu-Chen Liang +1 more
code/data similarity 0.65
Abstract · View our project page: https://ming053l.github.io/Doctor-Trigger/

Face swapping and generative editing enable realistic facial manipulation, while online redistribution complicates publisher oversight of circulating face images. Existing approaches establish provenance, detect manipulation, recover embedded watermarks, verify image integrity, or disrupt unauthorized face manipulation. These capabilities do not necessarily establish consistency with a specific publisher-enrolled release, limiting the evidence available for verification and human review. To address this gap, we introduce Doctor Trigger, a framework for verifying circulating faces against private publisher-held release records. Before publication, the publisher embeds a weak keyed Fourier-phase signal in each face while preserving visual fidelity and stores its spectral reference. Afterward, verification compares the circulating face with that reference and uses key projection to measure retention of the release-specific signal. Therefore, relation and key evidence jointly help publishers identify suspicious copies for review, potentially reducing the circulation of manipulated images. Experimental results across FF++, DFDC, and Celeb-DF show that spectral agreement separates tested face swaps and generative manipulations from enrolled releases, while the keyed signal distinguishes published versions from their unreleased originals under the evaluated key-preserving benign transformations. Doctor Trigger complements provenance metadata with release-specific face verification and record-linked evidence for further assessment.

0.40

MoviEdit: Long-form Multi-Shot Video Editing Made Easy

cs.CV cs.AIcs.LG ·2026-11-02 ·Suhyeon Lee, Zhengang Li, Soo Ye Kim, Peihao Zhu, Jong Chul Ye +4 more
benchmarkcode/datacommunity pick similarity 0.65
Abstract · 26 pages, 15 figures

Long-form multi-shot video editing requires an intended edit to remain coherent across abrupt changes in scene, viewpoint, and subject appearance, while processing substantially longer visual context than conventional single-shot editors. We introduce MoviEdit, an end-to-end framework for scalable long-form multi-shot video editing that jointly addresses the challenges of data, modeling, and evaluation. To provide cross-shot supervision data at scale, we develop automatic data-generation pipelines spanning synchronized multi-view footage, real-world multi-shot videos, and generated multi-shot content, yielding 87K paired editing videos without manual annotation. Architecturally, MoviEdit employs a cascaded bidirectional-autoregressive design: a lightweight bidirectional stage first establishes a globally coherent editing draft over a spatiotemporally compressed representation, which is then autoregressively propagated and refined over the full video. We further introduce MoviEdit-Bench, a dedicated benchmark for systematic evaluation of long-form multi-shot video editing. Extensive experiments show that MoviEdit substantially outperforms existing open-source models and performs on par with industry frontier models across text-guided and image-guided editing. Visit our project page for complete results and to access the benchmark: https://multi-shot-edit.github.io/

0.36

Can Tainted Pixels Expose Deepfake Videos?

cs.CR cs.CV ·2026-08-26 ·Juan Hu, Shaojing Fan, Sanjay Saha, Marc Herrera, Terence Sim
similarity 0.64
Abstract

Publicly-acceesible face-manipulation tools have made deepfake creation accessible to non-expert users. Against these, existing defenses are mostly post-hoc, detecting only after forgery has occurred, and operating on still images rather than videos. Research is lacking in i) the proactive protection of published facial videos against black-box manipulation tools, and in (ii) understanding its perceptual effect on human viewers. We introduce TaintedPixels, a proactive video-protection method built around an asymmetric visibility trade-off: the embedded watermark should remain inconspicuous in the published video but become obvious once a downstream tool manipulates the video. TaintedPixels injects structured periodic perturbations into the blue channel of facial regions and refines them under stripe-visibility, color-cast, and video-level LPIPS budgets, with lightweight motion-adaptive deployment. We believe TaintedPixels is the first proactive defense designed specifically against black-box manipulation tools rather than image-level pipelines or specific surrogate generators. Across three publicly available off-the-shelf video manipulation tools and two off-the-shelf detectors, TaintedPixels attains the highest forgery fake rate while keeping perturbations small (LPIPS = 0.0042). Our non-expert human study, conducted on a diverse set of 300 video stimuli spanning different lighting conditions, backgrounds, and skin tones, shows that protected source videos draw a 3.26% suspicion rate, while forgeries from protected sources are identified as fake much more often than forgeries from unprotected sources (90.72% vs. 56.71%). This validates the effectiveness of TaintedPixels.

0.25

VOR-Bench: A Human Perception-Driven Benchmark for Video Object Removal

cs.CV cs.AI ·2026-09-15 ·Haonan Huang, Tianrui Qiu, Xianghao Zang, Yinan Du, Zhixiang He +8 more
publishedbenchmarkdataset similarity 0.64
Abstract · BMVC-2026

Despite its crucial role in video object removal (VOR), existing evaluation paradigms face two critical limitations: questionable references and a misalignment between tradi- tional metrics and human preference. To address these challenges, we introduce VOR- Bench, which advances VOR evaluation through three integrated components. First, we present the VOR Dataset (VORD), the first benchmark dataset providing both paired edited videos and graffiti masks. Its unique strength lies in a diverse data spectrum, which encompasses model-generated, tool-rendered, and camera-captured data, ensuring robust assessment across real-world scenarios. Second, we develop rMPAF, a realistic Motion- capable Paired-video Acquisition Framework. By combining the strengths of image- based object removal and fine-tuned video generation models, rMPAF automatically generates realistic, motion-coherent paired videos. Finally, we propose three evaluation dimensions and introduce VOR-MDSM, the first perception-driven VLM-based scoring model specifically designed for mask-guided VOR. It bridges the gap between arithmetic metrics and human perception by covering the essential visual attributes and matching nuanced human judgment. Extensive experiments demonstrate that VOR-Bench yields evaluation results that align closely with human perception, achieving a remarkable cor- relation (\r{ho} > 0.9) with subjective assessments. We will release VOR-Bench along with its documentation to ensure full reproducibility.

0.40