CS Paper Radar

A Multimodal Framework for Measuring Creator Communication on Short-Form Video

Julie R. Ancis, Michael Fire, Omer Tarshish

cs.SI cs.CLcs.CY ·submitted 2026-10-16 10:41 UTC · v1

Abstract

Short-form video combines speech, imagery, and on-screen text, creating challenges for systematic content analysis. We present a five-stage multimodal framework for measuring creators' themes and communication strategies at scale. The framework integrates platform metadata, speech transcripts, and vision-language descriptions of sampled frames, then applies large language models to generate predefined indicators and open-ended labels. An audited keyword scheme consolidates thematic labels, while creator-weighted aggregation gives each account equal weight regardless of posting volume. We demonstrate the framework on 9,852 TikTok videos from sixteen creators, retaining 7,311 videos for analysis across seven themes and seven communication indicators. Automated profiles reveal substantial differences in creators' use of education, humor, and responses to current events. Validation against two independent expert annotators yielded a mean framework-expert agreement of Cohen's $κ= 0.40$ across the six binary indicators, close to the $κ= 0.42$ the two experts achieved with each other, though agreement varied substantially across individual indicators. The framework provides a modular approach to comparative analysis of creator communication, combining scalable annotation with transparent aggregation and construct-specific validation.

Comment
17 pages, 3 figures, 7 tables

Closest papers

Nearest neighbours by meaning across the whole index.

YTShorts-92K: An Interpretable Multimodal Dataset of YouTube Shorts

cs.MM ·2026-11-08 ·Celian Charrin, Sidonie Abuzakuk, Artúr Vlaszov, Nándor Birher, Roland Molontay
dataset similarity 0.69
Abstract · 10 pages, 3 figures, 3 tables

Short-form video has become one of the dominant formats of online content, yet large-scale datasets enabling the analysis of what makes these videos successful remain limited. Existing resources often focus on post-publication popularity signals or rely on non-interpretable representations such as high-dimensional embeddings. Here, we introduce YTShorts-92K, a dataset of 92,654 YouTube Shorts collected from 345 creators and spanning 12 manually defined content categories. Each video is enriched with an interpretable multimodal feature set covering metadata, textual information, visual characteristics from frames and thumbnails, and cross-modal consistency measures. We describe the collection methodology, data cleaning process, and feature extraction pipeline, followed by descriptive analyses of the diversity of creators, topics, and extracted features. This dataset offers a reusable resource for studying short-form video content, online attention, and multimodal content analysis.

0.30

What Does the Model See? Representation Validity in Multimodal LLM-Based Video Analysis

cs.CV cs.CL ·2026-11-03 ·Christian Pipal, Eva-Maria Vogel, Morgan Wack
benchmark similarity 0.65
Abstract

Multimodal large language models (MLLMs) allow social scientists to analyze video at scale, but researchers must decide which parts of a video to provide as input. We use the term representation validity to examine how text, audio, video, and combined inputs affect measurement and subsequent analysis. We compare input representations across six open-weight MLLMs and three human-annotated video datasets. No representation consistently performs best. On seven news-related constructs, native-video performance varies little across models (macro-F1 = .70-.77), while extracted-text performance ranges from .52 to .80. Three models perform better with text and three with video. Combining inputs rarely improves performance. Higher classification accuracy also does not reliably produce estimates closer to those based on human codes. The highest-F1 representation best reproduces human-coded regression coefficients in 17 of 42 comparisons. Even when each model uses its highest-F1 deployable representation, 45% of sample-prevalence estimates differ from the human-coded benchmark by more than ten percentage points. Input representation therefore requires validation for both the construct being measured and the quantity estimated from the resulting labels.

0.09

The 2026 World Cup, Anime Edition: Production, Engagement, and Circulation of Generative Memetic Sports Content

cs.SI ·2026-10-10 ·Azneef Chowdhury, Ho-Chun Herbert Chang
similarity 0.65
Abstract · 15 pages, 16 figures, including appendix Project Website: wcanime-mu.vercel.app

Generative AI gives audiences new ways to portray real sporting figures through familiar fictional worlds. Expanded opportunities to create, however, need not broaden cultural representation or distribute attention evenly. We examine this tension through the supply, demand, and circulation of anime style football videos on Instagram. Using 2,021 posts and associated video and account data, we combine caption entity extraction, human-reviewed visual identity clustering, engagement analysis, and content-reuse networks. Representation is uneven across athletes, while Blue Lock, Naruto, and One Piece account for 60.1% of explicit franchise mentions. Among posts referencing generative tools, rankings by total engagement differ from rankings by average engagement per video, distinguishing aggregate visibility from response to individual posts. Reuse is extensive: 772 of 1,548 evaluated videos belong to 124 detected groups, including 71 spanning multiple known accounts. Shared material connects a large account component alongside smaller groups, and later observed uploads receive lower median normalized engagement than first observed appearances. These findings characterize generative cultural crossover as a practice organized around concentrated cultural resources, uneven attention, and extensive recirculation. More broadly, they distinguish opportunities for visual authorship from the processes through which particular representations become widely visible.

0.38

Auditing Exposure to Harmful Content on TikTok using Multimodal Language Models: A Cross-National, Age-Stratified Study

cs.CL ·2026-08-18 ·Hamidreza Saffari, Francesco Pierri
similarity 0.65
Abstract · 20 pages, 16 figures, 14 tables

Online video platforms can expose young users to harmful content, but independent audits remain difficult because video annotation is costly and moderation judgments vary across languages. We audit TikTok in France, Italy, and Sweden with sockpuppet accounts representing four age personas (13, 16, 19, 40), collecting 36,971 videos from passive For-You-page scrolling and active sessions that scroll, search for harm keywords, and scroll again. To scale annotation, we validate four multimodal LLMs against native-speaker labels on a 300-video reference set. Gemini 2.5 Flash with eight sampled frames plus text performs best (aggregate kappa = 0.42), at half the per-call cost of native-video upload, and we apply it to a 10% sample for approximately \$50 in total API spend across both modalities. Keyword search returns 35-56% harmful content, a 1.5-7.5x increase over the scrolling baseline in ten of twelve country-age combinations; the spike is temporary and flattens the age differences observed in France and Sweden. Under passive scrolling, Italy has the highest harm rate at every age, with Italian age-19 reaching 48.6%. Overall, MLLM-based auditing offers a scalable approach for cross-national youth-safety audits, while provider safety filters (1.1% refusal rate) under-count the most explicit harms.

0.41

How Do Professional Editors Evaluate the Editing Quality of AI-Generated Cinematic Video Ads?

cs.HC ·2026-08-25 ·Po-Ming Law, Weizhi Li, Arpit Narechania
similarity 0.64
Abstract

On social media, we often encounter short-form video ads that employ cinematic editing techniques to evoke an emotional response. While AI tools are beginning to generate such cinematic ads automatically, we lack a fine-grained framework for evaluating these ads. In this paper, we first characterize social media video ad formats and identify cinematic ads as a recurring format in our corpus. We then analyze the duration, shot structure, audio and text elements, and editing techniques of cinematic ads to inform a two-stage generation pipeline in which an LLM first generates a shot plan and a video generation model renders the video. Using this pipeline, we generated 70 cinematic ads for 35 real brands and recruited professional video editors to critique their editing choices. From their critiques, we derive six dimensions of editing quality: narrative progression, audiovisual coordination and sound design, visual composition and graphics, shot-to-shot continuity, message and brand coherence, and temporal rhythm and pacing. We discuss how these dimensions can guide editing-aware generation, human evaluation, and automated evaluation of AI-generated cinematic ads.

0.16

BiliPicks: Seven Years of Weekly Platform Curation on Bilibili with Video Metadata, Comments, and Danmaku

cs.SI ·2026-10-28 ·Ruomeng Liu
similarity 0.63
Abstract

BiliPicks documents seven years of Bilibili's Weekly Picks, a platform-curated editorial video series. The dataset covers 371 publicly listed issues and 13,758 curated videos from 22 March 2019 to 7 May 2026. It links issue dates, themes and editorial recommendations to video metadata, 47,799,657 top-level comments, 125,690,760 time-synchronised overlay comments (danmaku), and 13,739 cover images. Weekly issues provide a consistent unit for tracing changes in category composition, recurring creators and editorial framing. Corresponding comments and danmaku connect video-level discussion with audience expressions tied to particular moments, while covers add a visual dimension. We evaluate retrieval coverage and illustrate longitudinal analyses of selection, audience language and cover presentation. BiliPicks provides a dataset for future studies of platform curation, linking changes in published selections to their editorial framing and corresponding audience engagement.

0.13

BanglaMemeX: Advancing Cultural Metaphoric Image Interpretation in Bangla with a Multimodal Explainable Dataset

cs.CL cs.CV ·2026-09-07 ·Md. Sadman Sakib, Zisan Mahmud, Md. Fahim Arefin, Md Fahim
top venuebenchmark similarity 0.63
Abstract · Accepted at EMNLP 2026 Findings - 37 pages, 17 figures, 19 tables, including appendices

Vision Language Models have achieved strong performance on multimodal benchmarks, yet their ability to reason about culturally grounded and metaphor-rich content remains insufficiently studied. Internet memes present a challenging setting where meaning emerges from implicit interactions between image, overlaid text, sarcasm, and shared socio-cultural knowledge rather than literal visual recognition. This challenge is amplified in low-resource languages such as Bangla, where code-mixing, stylized scripts, and culturally specific symbolism introduce substantial distribution shift. In this work, we introduce BanglaMemeX, a culturally grounded multimodal benchmark comprising 3,000 Bangla memes annotated with multi-dimensional labels (humor, sarcasm, offensiveness, motivational intent, and overall sentiment) and human-written explanations that explicitly describe textual and visual metaphors. We systematically evaluate modern VLMs on both classification and explanation generation, revealing that current models struggle to interpret implicit cultural cues despite reasonable surface-level accuracy. Our results highlight the need for culturally-aware multimodal systems capable of grounded reasoning under linguistic and cultural distribution shift.

0.29

Mind The Gap (MTG-AD): Dataset and Fine-grained Evaluation for End-to-End Inline Audio Descriptions

cs.CV cs.AIcs.HC ·2026-10-29 ·Subhashini Venugopalan, Jenny Tan, Taylor Roper, Jimmy Tobin, Anton Kast +3 more
top venuebenchmarkdatasetcode/data similarity 0.63
Abstract · NeurIPS 2026 Evaluations & Datasets Track, camera ready

Audio descriptions (AD) are essential for making visual media accessible to blind and low-vision (BLV) users. While Multimodal Large Language Models (MLLMs) offer a scalable solution for generating audio descriptions on-demand, their performance on ``inline'' audio descriptions -- which must fit within existing silences in a video -- remains under-explored, particularly for diverse user-generated content (UGC). We present a comprehensive investigation into MLLM-generated inline audio descriptions. First, we introduce MTG-AD, a professionally annotated dataset of 36 hours of videos, each averaging $\sim$5 mins. and 8k+ descriptions in total, co-created by paired blind and sighted AD professionals to provide a high-quality benchmark for this task. Second, we propose MTG-AD-Eval, an evaluation framework based on audio description expert guidelines that provides fine-grained, reference-free actionable metrics for model improvement. Crucially, our framework evaluates the end-to-end task: it renders the generated scripts into audio tracks to assess their fit within the original video's natural silences, minimizing disruptive overlaps. Our analysis across frontier MLLMs reveals that while models accurately describe visual events and substantially reduce speech overlaps, gaps persist in narrative pacing and flow. Finally, we empirically validate that our automated and reference-free metrics correlate significantly with human quality ratings from sighted raters and 7 BLV participants, finding that while users narrowly prefer human-authored audio descriptions, they find MLLM-generated audio descriptions highly beneficial for comprehension. We provide actionable system design guidance and release MTG-AD at https://hf.co/datasets/google/mind-the-gap-inline-ad and MTG-AD-Eval at https://github.com/google/mtg-ad.

0.40