CS Paper Radar

ChildSafeAds Shared Task 2026: Commercial Content in Child-Facing YouTube Videos

Thales Bertaglia, Catalina Goanta, Gerasimos Spanakis, Gunes Acar

cs.CL cs.CY ·submitted 2026-08-19 17:44 UTC · v2
unusual

Abstract

ChildSafeAds is a shared task on commercial content in YouTube videos likely to reach children and teenagers. The dataset contains 3,360 videos from 939 channels, each starting with a sponsor segment submitted to SponsorBlock. We pair each segment with its transcript, video, and channel information, plus a sales or service page linked from the video description. Systems determine the type of offer being promoted (ST1), assign product categories (ST2), and identify potential legal risks (ST3). Evidence is divided into four cumulative access levels, from the transcript to the linked page, so performance can be compared against the cost of collecting additional data. YouTube's "Includes paid promotion" label was absent from 45.5% of the videos. GPT-5.4 produced the labels after the organiser team, including a legal expert, reviewed samples and iterated on the taxonomy, prompts and model choices. GPT-5.6-luna independently labelled the development set to measure cross-model agreement. The evaluation received 21 entries; the highest mean macro-F1 was 0.735, while the best transcript-only entry reached 0.652. Across four system papers, the linked promotional page mainly helped identify what was being sold and added little to compliance-risk classification. The competition also exposed sensitivity to rare labels, advertiser overlap across splits and substantial variation in the ST3 labels.

Closest papers

Nearest neighbours by meaning across the whole index.

Nürnberg NLP at ChildSafeAds 2026: Structurally Dissimilar Voter Ensembles under Four Levels of Data Access

cs.CL cs.LG ·2026-09-28 ·Philipp Steigerwald, Eric Rudolph, Jens Albrecht
similarity 0.65
Abstract · Accepted at the ChildSafeAds 2026 Shared Task @ NLLP Workshop, EMNLP 2026 (1st place in 2 of 3 subtasks)

We describe the Nürnberg NLP system for ChildSafeAds 2026. The shared task asks what a monitoring system for commercial content in child-facing YouTube videos can achieve at a given level of data access. We answer with per-subtask ensembles of nine voters, organised into three branches that differ in backbone, adaptation method and class scope. Selection rests on channel-disjoint cross-validation, with the development set as a transfer check. The system wins two of the three subtasks. Its product-category score (ST2, 0.8243) and its compliance-flag score (ST3, 0.6530) are the best of the 22 final entries, and it places third on the task mean (0.7079). We further compare four access levels and report the cost at test-set scale.

0.49

Auditing Exposure to Harmful Content on TikTok using Multimodal Language Models: A Cross-National, Age-Stratified Study

cs.CL ·2026-08-18 ·Hamidreza Saffari, Francesco Pierri
similarity 0.53
Abstract · 20 pages, 16 figures, 14 tables

Online video platforms can expose young users to harmful content, but independent audits remain difficult because video annotation is costly and moderation judgments vary across languages. We audit TikTok in France, Italy, and Sweden with sockpuppet accounts representing four age personas (13, 16, 19, 40), collecting 36,971 videos from passive For-You-page scrolling and active sessions that scroll, search for harm keywords, and scroll again. To scale annotation, we validate four multimodal LLMs against native-speaker labels on a 300-video reference set. Gemini 2.5 Flash with eight sampled frames plus text performs best (aggregate kappa = 0.42), at half the per-call cost of native-video upload, and we apply it to a 10% sample for approximately \$50 in total API spend across both modalities. Keyword search returns 35-56% harmful content, a 1.5-7.5x increase over the scrolling baseline in ten of twelve country-age combinations; the spike is temporary and flattens the age differences observed in France and Sweden. Under passive scrolling, Italy has the highest harm rate at every age, with Italian age-19 reaching 48.6%. Overall, MLLM-based auditing offers a scalable approach for cross-national youth-safety audits, while provider safety filters (1.1% refusal rate) under-count the most explicit harms.

0.41

ECP-Bench: Benchmarking and Learning Entertainment Content Promotion with Foundation Models

cs.IR cs.LG ·2026-08-26 ·Hyomin Kim, Bowen Chen, Jin Huang, Zhao Wang, Qiaozhu Mei +1 more
benchmark similarity 0.52
Abstract

Content promotion spans a broad set of skills, from understanding content to forecasting its market reception. However, LLMs' ability to support such promotion decisions remains underexplored. Existing studies are often limited to a single task (e.g., popularity prediction) or a small set of tasks within a single domain (e.g., movies). As a result, there is a lack of understanding of LLMs' abilities in the full promotion process and how these abilities generalize across different tasks and domains. In this work, we introduce ECP-Bench, a benchmark containing 1.9M movie, game, and music items and 423,451 questions across 33 tasks in five content-promotion skill families. Our evaluation shows that frontier models achieve only 51.9\% overall accuracy and lose much of their advantage on post-cutoff content, with drops of up to 19.1 percentage points. In contrast, open-weight models fine-tuned on ECP-Bench achieve up to 60.3\%, remain substantially more stable across the knowledge cutoff, generalize to unseen content and tasks, and exhibit meaningful cross-domain generalization.

0.17

FIRM-Video: Check Before You Score for Reliable Text-to-Video Reward Modeling

cs.CV ·2026-08-22 ·Peiyuan Zhang, Xiangyu Zhao, Hongbo Liu, Xiaoxing Hu, Mingxin Liu +6 more
similarity 0.51
Abstract

Reliable reward models are essential for text-to-video evaluation and alignment. However, the trade-off between evaluation accuracy and inference efficiency places high demands on the quality of training supervision. Existing approaches often rely on holistic judges with fixed rubrics or open-ended reasoning, leading to incomplete inspection, unfaithful justification, and entangled attribution. We introduce FIRM-Video, a unified checklist-driven data construction framework based on a check-before-score principle: construct dimension-specific checklists, verify each criterion against temporal visual evidence, and aggregate only verified decisions. For Instruction Following, FIRM-Video decomposes prompts into weighted atomic requirements; for World Coherence, it constructs prompt-calibrated, target-specific checks grounded in visible entities and actions; and for Perceptual Quality, it applies a generic taxonomy of visual defects. The verified criteria and scores are further transformed into natural-language analyses for end-to-end reward modeling. Subsequently, we construct FIRM-Video-90K with 88,044 dimension-specific instances from 29,348 videos, and introduce FIRM-Video-Bench with 750 point-wise human annotations across 250 videos. The Qwen3-VL-based FIRM-Video-8B achieves the best overall MAE on FIRM-Video-Bench while consistently delivering the highest VBench Total, Quality, and Semantic Scores in Best-of-8 sampling across three video generators.

0.42

AdSpark: A Large-Scale Dataset and Benchmark for Product-Centric Advertisement Video Generation

cs.CV ·2026-10-07 ·Zhifei Yang, Zhao Jiang, Keyang Lu, Honghe Zhu, Zheng Zhang +4 more
benchmarkdataset similarity 0.51
Abstract

Product-centric advertisement video generation aims to create promotional videos that preserve fine-grained product identity while presenting selling points through coherent multi-shot narratives. However, this emerging task remains underexplored due to the lack of large-scale advertisement-specific datasets and comprehensive evaluation frameworks. To address this gap, we introduce \textbf{AdSpark}, a large-scale dataset and benchmark for product-centric advertisement video generation, based on data from a major e-commerce platform. \textit{AdSpark-300K} contains approximately 300K reference image--prompt--video triplets, comprising a real-world subset and a synthetic subset. Each sample provides structured advertisement annotations, including product identity annotations, selling-point descriptions, creative plans, and aligned audio scripts, enabling models to learn product preservation and advertisement-oriented visual storytelling. We further propose \textit{AdSpark-Bench}, a diagnostic benchmark that evaluates generated advertisements across six dimensions, including visual quality, product fidelity, instruction adherence, temporal coherence, audio alignment, and advertisement effectiveness. Based on AdSpark-Bench, we evaluate representative models, revealing key challenges in product preservation, multi-shot storytelling, and selling-point visualization. Experiments with AdSpark-300K-finetuned models further validate the effectiveness of our dataset. AdSpark provides a unified dataset and benchmark for future research, and we will release the dataset upon acceptance.

0.15

Beyond Risk Cues: A Mechanism-Level Benchmark and Verifiable Debate for Scam Video Detection on the Web

cs.AI ·2026-11-04 ·Yifan Wang, Junyu Lu, Shun Zhang, Chaozhuo Li, Fanliang Bu +4 more
benchmarkcode/data similarity 0.49
Abstract

Online scam videos pose significant threats to individuals and society. Existing studies primarily focus on binary classification, offering limited insights into how scams work and the reasoning behind detection decisions. Moreover, they often overlook the challenge of distinguishing scam videos from superficially similar non-scam content, such as advertisements, and the robustness of detection models against misleading arguments. These limitations hinder reliable scam video detection. To address these limitations, we introduce MechScam, a benchmark comprising 6,614 videos covering diverse scam mechanisms and non-scam categories. MechScam provides explanations supported by video evidence, annotations of potential scam elements and their context, and misleading arguments constructed from genuine evidence. Recognizing that genuine evidence can still be used to justify misleading conclusions, we further propose ScamCourt, a debate framework in which trained prosecution and defense debaters present opposing arguments, while a trained judge makes decisions based on a transcript with verified citations. Extensive experiments show that state-of-the-art MLLMs struggle to distinguish scam videos from similar non-scam content. ScamCourt outperforms larger MLLMs and same-backbone fine-tuned baselines, reducing false positives and showing stronger resistance to misleading arguments. Together, MechScam and ScamCourt contribute to more reliable and interpretable scam video detection. The dataset and code are available at https://github.com/SuperYFan/MechScam.

0.33

A Multimodal Framework for Measuring Creator Communication on Short-Form Video

cs.SI cs.CLcs.CY ·2026-10-16 ·Julie R. Ancis, Michael Fire, Omer Tarshish
similarity 0.48
Abstract · 17 pages, 3 figures, 7 tables

Short-form video combines speech, imagery, and on-screen text, creating challenges for systematic content analysis. We present a five-stage multimodal framework for measuring creators' themes and communication strategies at scale. The framework integrates platform metadata, speech transcripts, and vision-language descriptions of sampled frames, then applies large language models to generate predefined indicators and open-ended labels. An audited keyword scheme consolidates thematic labels, while creator-weighted aggregation gives each account equal weight regardless of posting volume. We demonstrate the framework on 9,852 TikTok videos from sixteen creators, retaining 7,311 videos for analysis across seven themes and seven communication indicators. Automated profiles reveal substantial differences in creators' use of education, humor, and responses to current events. Validation against two independent expert annotators yielded a mean framework-expert agreement of Cohen's $κ= 0.40$ across the six binary indicators, close to the $κ= 0.42$ the two experts achieved with each other, though agreement varied substantially across individual indicators. The framework provides a modular approach to comparative analysis of creator communication, combining scalable annotation with transparent aggregation and construct-specific validation.

0.15

Persistent Identity in Person Tracking: Online Enrollment and Registered Recognition

cs.CV ·2026-11-18 ·Ratha Siv, Matei Mancas, Bernard Gosselin
similarity 0.48
Abstract · 7 pages, 2 figures, 7 tables. An earlier version of this paper was submitted to an IEEE conference

We extend PoseTReID with online appearance-bank matching (Gallery) and a residual-conflict guard for registered SVM recognition. All methods use supplied ground-truth (GT) person boxes. We average all-box assessment IDF1/HOTA equally across videos. On six SportsMOT clips, StrongSORT leads standalone trackers (79.44/84.72%) and exceeds both Gallery variants. Over eleven videos, Gallery AIN (83.29/85.44%) and COPE (84.66/86.38%) exceed standalone trackers with complete results. This lead depends on two retrospectively added GTA development compilations and is absent on the original nine videos. Registered SVMs lead both cohorts among complete results, with AIN at 95.72/94.82% over eleven videos. They use every GT crop from approximately 20% of frames, including future appearances. On MOT20-03, registered AIN reached 96.80% IDF1 and assigned the correct enrolled name to 90.72% of assessment GT observations. Gallery and standalone trackers receive no per-video labelled enrollment. Both identity routes restore identities across internal title cards, but Gallery trails SORT on three videos (four in HOTA) and both SVMs trail SORT on MOT16-04/MOT20-03. Under six-clip persistent-only scoring, native Gallery gains 2.39 percentage points in mean IDF1 over a matched control, with lower correct-P coverage and 5.02 times as many encoder attempts. Pose gating is mixed; the guard's metric effect is not isolated.

0.26