CS Paper Radar

Synthetic Realities: Evaluating Human Ability to Distinguish AIGenerated Videos from Real Footage

Danyal Sarfraz, Raja Mubashar Karim, KwanMyung Kim

cs.CV cs.HC ·submitted 2026-11-01 15:26 UTC · v1
published

Abstract

Recent advances in generative AI have enabled multiple pathways for high-fidelity video synthesis: text-to-video, image-to-video animation, and video outpainting. Empirical side-by-side evaluations of how untrained viewers perceive and distinguish these outputs from real footage remain scarce. In this study, we systematically compare human detection accuracy across these three AI generation techniques within three thematic contexts: historical footage, film and media content, and natural environments. We constructed a balanced stimulus set comprising equal amounts of real and AI-generated videos (18 each). The AI clips were evenly distributed across the three generation methods using Google's Veo 3, Lightricks LTX Video, and Wan VACE. All videos were produced and standardized within the ComfyUI framework to ensure consistent quality and duration. 87 participants judged every clip in a binary forced-choice task ("real" vs. "AI-generated"). Participants correctly identified videos 60% of the time on average. Image-to-video clips were recognized most accurately (79%), followed by real footage (64%), outpainting (49%), and text-to-video (43%). Accuracy also varied by theme: film and historical scenes yielded higher detection rates than environmental clips, which were frequently mistaken for AI. Logistic regression confirmed significant effects of both technique and theme as well as their interaction (p < 0.001), indicating that detection success depends jointly on how the content was generated and what it depicts. Findings reveal a consistent bias toward assuming synthetic origins and highlight that perceptual realism in AI video is shaped more by context than by model type, underscoring the importance of media-literacy approaches and context-aware evaluation tools for navigating increasingly synthetic visual media.

Comment
Published in Proceedings of IASDR 2025
DOI
10.21606/iasdr.2025.1050

Closest papers

Nearest neighbours by meaning across the whole index.

Reality Perception Gap: Are AI-Manipulated Video Detectors Ready for the Real World?

cs.CV ·2026-11-11 ·Xiangrui Liu, Xinmiao Luan, Xinyuan Wang, Zhiyuan Fang, Chi-Yao Huang +4 more
benchmark similarity 0.70
Abstract

AI-manipulated video detectors are typically evaluated on clean, curated datasets. In deployment, however, videos have usually passed through repost compression on social-media platforms and sometimes were captured by imperfect hardware. To measure detection under these conditions, we introduce Reality Perception Gap (RPG), a benchmark of 10,000 videos from frontier generators and partially synthetic edits, processed with cross-platform repost compression and paired with real footage that carries camera-side optical artifacts. RPG also includes socially fragile content, such as crime and wartime footage, where fabricated videos can cause severe social harm. Across 18 detectors, none catches more than 15% of fakes at a 1% false-positive rate. A positive control and a re-encoding sweep separate two sources of failure: generator and content shift degrades detectors before any compression, and repost compression erases the low-level traces that the artifact-based detectors relies on. Existing detectors therefore struggle when deployment conditions strip away the generation artifacts they rely on. These complementary failures suggest that robust detection requires training under deployment conditions and combining low-level, temporal, and semantic evidence, where passive detection may have reached its limits.

0.19

HumanCentric-Scenes-AI: A Human-Curated Multimodal Dataset of AI-Generated Human-Centric Scenes Across Surveillance, Occupational Portraits, AI-Edited Images, and Short-Form Video

cs.CV cs.AIcs.HC ·2026-10-21 ·Nima Kamali, Sana Nasiri, Negar Kamali
datasetcode/data similarity 0.70
Abstract · 14 pages, 3 figures, 10 tables. Dataset: https://huggingface.co/datasets/Nima0Kamali/humancentric-scenes-ai

We introduce HumanCentric-Scenes-AI, a human-curated dataset of 296 AI-generated stimuli that depict people across four contexts in which people encounter AI-generated media. The Surveillance subset contains 85 AI-generated CCTV-style images. The Occupations subset contains 128 AI-generated occupational portraits in a balanced paired design covering 64 occupations with one female and one male version of each. The AI-edited subset contains 27 real photographs in which a region was altered or replaced by a generative model. The Video subset contains 56 AI-generated short videos. We curated the images and videos manually over multiple rounds to make them as photorealistic as possible. For each stimulus we release the file, the generation prompt where recorded, and per-stimulus metadata. We report depicted-gender distributions from an initial set of 134 generations without a specified gender and compare them with U.S. Bureau of Labor Statistics occupational gender shares. We report edit-region size and its association with zero-shot detection difficulty for the AI-edited subset. We report image-quality distributions for the Surveillance subset and subject-category coverage for the Video subset. HumanCentric-Scenes-AI is intended first for human-perception research, and also for calibration of AI-detection classifiers and bias analysis of generative models.

0.42

DF26: We Cannot Tell Fake From Real Anymore

cs.CV ·2026-09-07 ·Severyn Shykula, Andrii Yermakov, Ivan Samarskyi, Dmytro Mishkin, Jan Cech +1 more
benchmark similarity 0.70
Abstract

We introduce DF26, a novel benchmark for detecting AI-generated videos containing fully synthetic clips produced by recent text-to-video and image-to-video models. The videos capture single-person public-speaking scenarios, spanning direct-to-camera recordings, official statements, and studio interviews - 271 real and 2,420 synthetic videos generated by seven modern video models. The study on DF26 shows that human performance in detecting AI-generated videos, as well as state-of-the-art deepfake detectors, is close to random chance. Our results highlight the limitations of current evaluation protocols and motivate the need for benchmarks that explicitly measure robustness to modern generative model distribution shifts.

0.44

Beyond Source Prompts: Towards Visually Grounded AI-Generated Video Quality Assessment

cs.CV ·2026-10-15 ·Kehao Zhou, Yixuan Li, Yuxuan Jiang, Fan Zhang
code/data similarity 0.68
Abstract

Recent advances in generative artificial intelligence have accelerated video generation technologies. However, AI-generated videos (AIGVs) remain susceptible to various distortions, highlighting the need for dedicated video quality assessment methods. Content semantics play an important role in AIGV quality assessment, with source prompts typically used as textual references. However, these prompts may not accurately reflect the generated video content, resulting in a discrepancy between the intended textual description and the generated result. To address this limitation, this paper proposes VGCap, an AIGV quality assessment framework guided by visually grounded captions that align better with generated content. Specifically, we introduce two forms of captions: a global caption obtained by minimally revising the source prompt based on multi-frame visual evidence, and prompt-independent segment captions that describe localized temporal content. Together, these captions provide complementary textual evidence for assessing spatial quality, temporal quality, and text-video alignment. Experimental results on the LGVQ and FETV datasets show that VGCap outperforms representative baselines across all three dimensions. Further ablation studies on caption sources and temporal components confirm the effectiveness of visually grounded captions for each quality dimension, while highlighting the benefits of segment-level features for temporal quality prediction. The source of VGCap will be available at https://github.com/lolo123-123/VGCap.

0.32

Evaluating Visual Realism of AI-Generated Videos with Eye Tracking

cs.HC cs.AIcs.CV ·2026-10-28 ·Danqing Shi, Lan Jiang, Jake Lyons, Miri Zilka
similarity 0.68
Abstract

Human evaluation of AI-generated videos has so far relied on explicit human feedback, such as judgments or ratings. While these signals capture the conclusions viewers reach, they fail to offer insights into the perception that leads to them. This paper investigates eye tracking as a complementary human evaluation signal for AI-generated videos. We conducted two Visual Turing Test (VTT) experiments with a total of 90 participants on videos generated by latest generative models, in which participants watched each video twice and then gave their feedback. Across the experiments, we collected 4,286 valid eye-tracking trials on 438 videos. Our analysis indicates that gaze patterns offer informative evaluation signals that align with explicit human feedback. Building on the findings, we propose the first computational method GazeEval to predict the perceived visual realism of AI-generated videos from eye-tracking data. It achieves 71.3% accuracy in prediction, providing evidence that eye tracking data carries informative signals about how people evaluate a video. We release the study data to support further research on eye tracking for evaluating AI-generated video.

0.10

SCRiD: Spatial and Contextual References for Generalizable AI-Generated Video Detection

cs.CV ·2026-11-18 ·Hao Tan, Zekai Zuo, Boyang Chen, Jinghui Qin
similarity 0.68
Abstract · 14 pages,10 figures

Generalizable AI-generated video detection remains challenging because natural motion and content variations can resemble synthesis artifacts, making isolated video discrepancies unreliable indicators of authenticity. To address this challenge, we propose Spatial and Contextual Reference-informed Detection (SCRiD), a framework that interprets video variations through complementary spatial and contextual references. Specifically, we introduce Matched-Coordinate Residual Response (MCR) to characterize temporal residual associations under motion and coordinate references with matched subpixel sampling conditions. We further develop Real-Conditioned Score Adaptation (RCSA), which learns a content-conditioned denoising reference from authentic videos and probes its responses across multiple noise scales and quality views to guide feature adaptation. To integrate these complementary sources of evidence, we design Reversible Evidence Exchange (REE) to enable information exchange through reversible symplectic updates. We train SCRiD using synthetic videos from only 4 generators and evaluate it across 88 generator or task subsets spanning 6 benchmarks. Our method achieves average ACC and AP of 72.72% and 74.23%, respectively, outperforming the strongest baseline by 3.24 and 0.81 percentage points.Ablation and robustness experiments further validate the effectiveness of the proposed components and the stability of SCRiD under common post-processing operations.

0.30

TRACE: Trajectory Representation and Consistency Estimation for AI-Generated Video Detection

cs.CV ·2026-09-22 ·Huangsen Cao, Hongkang chu, Siyao Yu, Xin Ding, Jianfeng Dong +1 more
similarity 0.68
Abstract

Recent advances in generative video models have enabled the synthesis of visually realistic content, posing significant challenges to synthetic video detection. Existing detectors often rely on appearance artifacts, semantic inconsistencies, and temporal patterns that may be generator-specific, limitating generalization to unseen synthesis models. We investigate whether responses to a pretrained generative model provide more transferable forensic cues. Our key observation is that real and AI-generated videos exhibit distinct \emph{velocity responses} under a pretrained Flow Matching video model. This distinction persists when different pretrained video-generation backbones are used as probes, suggesting that velocity responses offer transferable forensic signals beyond visual artificts. Motivated by this observation, we propose \textbf{TRACE} (\emph{\underline{T}rajectory \underline{R}epresentation \underline{a}nd \underline{C}onsistency \underline{E}stimation}), a generation-process-aware framework for AI-generated video detection. TRACE leverages a pretrained video DiT as a velocity-field probe to extract representations at multiple flow time points, and models cross-frame consistency through velocity differences between adjacent frames. We further introduce a \emph{Real-Centered Trajectory Optimization} objective that encourages generator-invariant representation learning. Extensive experiments on AIGVDBench demonstrate that TRACE generalizes effectively across diverse generators, substantially outperforming prior state-of-the-art methods on unseen open- and closed-source video generation models.

0.14

WanPE: Towards Cinematic Prompt Enhancement for Modern Text-to-Video Generation

cs.CV ·2026-09-24 ·Yubo Zhu, Yawen Shao, Ziyun Dai, Zixun Fang, Kai Zhu +25 more
benchmarkhighly cited similarity 0.68
Abstract

Video generation begins in text space by authoring a cinematic screenplay, then materializes into pixels. As contemporary video generators scale to 30 seconds and faithfully follow complex conditions, the textual prompt largely directs the production, planning how actions, camera trajectories, lighting, and sound unfold across multi-shot sequences. In this paper, we present WanPE, a 397B-parameter prompt enhancement model trained on 1.05M real-world videos to master director-level cinematic planning. WanPE formulates shot-level cinematic plans via video-grounded reverse construction and employs Semantic-Consistency GRPO (SC-GRPO) to faithfully preserve user requirements across shots and over time. To benchmark this capability, we curate WanPEval, a human-annotated testbed covering durations from 5 to 30 seconds across varying intent granularities, supported by approximately 11K blind pairwise assessments. When powering Wan3.0's video generator, WanPE-397B boosts human preference over raw user prompts by 10.66-18.84 points at 5-15 seconds and by a dramatic 50.86 points in the 30-second arena. Ablation studies show that reverse construction demonstrates clear superiority over forward rewriting, while SC-GRPO robustly preserves semantic fidelity across model scales. Ultimately, WanPE leads all evaluated commercial offerings at 5-15 seconds and remains competitive with Seedance 2.5 at 30 seconds.

0.36