CS Paper Radar

Floor It, Then Fill It: Simulating Creator Ecosystems for Fair Video-Content Recommendation Systems at Billion User Scale

Sohini Roychowdhury, Long Sha, Sunand Agarwal, Payal Kamboj, Shuyang Chen, Junlin Zhang, Scott Chen, Hui Zhang

cs.LG ·submitted 2026-10-28 01:41 UTC · v1

Abstract

Engagement-optimized content recommendation systems exhibit a well-documented rich-get-richer dynamic that structurally disadvantages emerging creators on large-scale platforms. Evaluating debiasing interventions at scale demands faithful simulation of creator-viewer ecosystems, yet the fidelity, diversity, and validity of such simulators remain underexplored. We present a grounded user-simulation framework that models a heterogeneous population of ${\sim}42$K creators and their coupled viewer interactions over 12~weeks, and use it to evaluate equitable view-allocation policies formulated as a budgeted multi-armed bandit (MAB). The simulator captures three behavioral dimensions: (i)~\textbf{creator diversity}, modeled via a clustering approach grounded in real platform creator-level distributions; (ii)~\textbf{behavioral fidelity}, captured through coupled creator-content state machines that represent growth trajectories, engagement accumulation, and graduation dynamics; and (iii)~\textbf{population-level validity}, measured by calibration against observed platform metrics. We evaluate simulative-configurations that vary content-viewership target definitions, reward functions, and a small-creator reserve lever to maximize creator-content recommendation equity. A floor-protected per content-view-target with a view-parity-gap reward guarantees 90\% of view targets being met by creators every week, grows small-creator organic reach ${\sim}{+}33\%$ per week, and achieves high distribution stability (set of creators with funded-views Jaccard $= 0.819$). We analyze the Sim2Real gap and propose validation protocols, contributing evidence for when simulation-derived fairness conclusions can be trusted.

Comment
9 pages, 5 figures, Neurips Workshop 2026

Closest papers

Nearest neighbours by meaning across the whole index.

CUEing User Simulators: Calibrated User Embeddings for Multi-Turn Benchmarking

cs.CL ·2026-10-01 ·Anjali Kantharuban, Jonas Mueller
similarity 0.64
Abstract

Recent benchmarks rely on user simulators to evaluate AI agents in multi-turn interaction. While existing simulation techniques demonstrate surface fidelity to human style and behavior, ecologically valid interactive benchmarking also requires alignment in when and how agents fail across simulated and real user populations. We find that existing simulators lack outcome calibration: agreement with observed success rates and failure patterns when real users interact with the same agent. We introduce Calibrated User Embeddings (CUE), a framework that both encodes observed sessions and samples continuous representations, then decodes them into persona commands to steer LLMs to act as user simulators without training. Through this, we evaluate user-conditioned replay of past sessions and aggregate metric agreement when sampling novel personas for the same tasks. On $τ^2$-Bench, CUEd simulators commit fewer simulator-attributed errors and more faithfully reproduce real-user agent failure modes, aggregate success rates, and outcomes for specific task-user pairs than other persona-based simulation methods. These gains coexist with competitive user fidelity as measured using metrics established in prior work. After being fit to mostly customer support interactions, the same CUEd simulators generalize to document creation, math tutoring, and casual conversation, and remain effective across different simulator LLMs without CUE retraining.

0.13

Challenges and Solutions for Bandits in the Wild: Warm-Started Mixture Bandits for Cross-Cohort Slate Recommendation

cs.LG cs.AI ·2026-09-29 ·Serafima Lebedeva, Sumantrak Mukherjee, Ali Arshad Sadal, Ilias Ekşi, Rahul Sharma +6 more
similarity 0.63
Abstract · 11 pages, 3 figures, preprint

Many recommender services repeatedly encounter cold-start cohorts, where new users arrive with little or no interaction history. This creates two challenges: learning user preferences quickly from limited feedback and sustaining useful recommendations when each user has a finite catalog that can become repetitive or depleted over time. We propose CohortMix-TS, a warm-started mixture bandit that learns latent user groups from earlier cohorts and uses available metadata to construct group-informed priors for new users. Starting from these fixed priors, the model personalizes independently as feedback from each user becomes available. Session slates combine Thompson sampling with diversity and inventory-depletion controls. We evaluate CohortMix-TS through simulation, semi-synthetic experiments, and a 25-day randomized in-the-wild deployment with 713 registered participants in a Campus Games quiz application. Our evaluations show that cross-cohort transfer improves early recommendation quality and user-level regret, while inventory-aware slate construction helps prevent premature exhaustion of preferred items. In the field deployment, treatment users also showed a larger early-to-late change in correctness than users receiving random recommendations. Together, these results show how warm-start transfer and inventory-aware recommendations can support personalization for short-lived, repeatedly cold-starting cohorts.

0.27

"If It Looks Like a User": Measuring Real-Time Moderation Effects via Social Media Simulation

cs.SI cs.CYcs.MA ·2026-08-17 ·Enrico Verdolotti, Gianluca Nogara, Luca Luceri, Silvia Giordano
publishedtheory similarity 0.62
Abstract · 15 pages, 4 figures, 2 tables. Accepted for presentation at the Social Simulation Conference (SSC) 2026

Agent-based social media simulators offer a controlled environment to study content moderation, yet their value hinges on how faithfully they reproduce real platform dynamics. We develop a calibrated extension of SimSoM, an agent-based model of information diffusion on social networks, grounded in a real-world dataset of online vaccine discourse during the COVID-19 pandemic. Our approach replaces ad-hoc parametrisations with empirically fitted distributions, optimised via CMA-ES (Covariance Matrix Adaptation Evolution Strategy) and validated against real data across temporal, distributional, and structural dimensions. Using this validated simulator, we provide three key contributions. First, we show that the calibrated model reproduces key statistical signatures of the empirical data, including activity distributions, post/reshare ratios, and temporal patterns. Second, we apply established misinformation-spreader detection and prevention methods to both empirical and simulated data, progressively removing top-ranked users and showing that the resulting decline in low-quality content is consistent across the two. Third, comparing static (retroactive) and dynamic (in-simulation) moderation across 30 network realisations, we show that static evaluation significantly overestimates the effectiveness of user bans for the most effective detectors: when moderation is applied in real time, compensatory resharing by the remaining users dampens the expected reduction in low-quality content, so static estimates should be read as an upper bound. These findings highlight the necessity of simulation-based evaluation for content moderation policies and contribute a reusable, empirically grounded simulation framework.

0.26

OranSim: Simulating Consumer Response to Social Media Campaigns Before Launch

cs.SI ·2026-09-23 ·Jianxiang Ma, Mingfu Zhang, Xiaocui Yang, Yichen Gao, Junzhao Huang +1 more
code/data similarity 0.61
Abstract · 26 pages, 5 figures. Updated title and revised manuscript. Code: https://github.com/OranAi-Ltd/oransim

Social simulation studies how individual behavior and social interaction produce collective outcomes. In social media marketing, campaign actions shape which consumers encounter the content and how they respond; these responses then spread through the population. We propose OranSim, a social simulation framework that connects creative, creator, targeting, and budget choices to this process. Heterogeneous consumers receive exposure according to content matching and platform allocation and generate initial responses, which propagate among 60 population segments. Candidate campaigns share the initial population and aligned random numbers, making their response trajectories comparable under action changes. In a controlled synthetic campaign, doubling the budget approximately doubles reach while lowering mean content match and engagement probability among the reached consumers; mean 14-day cumulative simulated response mass rises to 1.96 times the baseline. LightGBM predictors fitted to 39,000 historical RedNote notes estimate platform engagement with log-scale $R^2$ of 0.56--0.62 in five-fold cross-validation; a separate 12,154-note corpus supplies temporal, unseen-creator, and held-out-niche test splits. Public-data experiments evaluate policy-value estimation and audience ranking, and paired synthetic outcomes test counterfactual scoring. Together, scenario trajectories and engagement estimates support campaign selection according to a prespecified marketing objective. Code is available at https://github.com/OranAi-Ltd/oransim.

0.73

Content Exploration Beyond the Feed: Creator Supply and the Shared Corpus

cs.IR cs.LGecon.EMstat.AP+1 ·2026-08-29 ·Yuanyuan Shen, Yiren Yan, Wenjie Li, Chunhui Zhu
similarity 0.61
Abstract

Industrial recommenders give new content initial views through budgeted exploration, then use early performance to decide further delivery. On many short-video platforms, exploration is the primary way new videos reach viewers. Viewer-side tests measure consumption; the published budget objectives we review omit creator response. We analyze four experiments on a major short-video platform. An eight-month creator ablation finds production exploration raises videos posted per creator by 8.55% and creators posting at least once by 7.10% relative to a minimal floor. A budget-matched reallocation raises creator participation with no detectable short-run viewer-side change. A year-long viewer ablation finds 1.74% more video views but 2.13% less view time. A delivered view creates immediate feed value, can trigger organic take-up, and can induce creator supply. Take-up and supply replenish a shared corpus, creating two measurement limits. Viewer-side A/B tests cancel the corpus effect when both arms consume the same corpus. Giving each arm its own corpus avoids cancellation, but turnover still controls the horizon. If the corpus turns over at rate w per posting cycle, a t-cycle experiment expresses at most wt of the eventual corpus effect. More users reduce noise but do not speed turnover. Before the corpus path visibly bends, data cannot distinguish a modest fast effect from an arbitrarily large slow one, so a valid confidence interval may lack a finite upper endpoint. As predicted, the three-week co-diverted experiment cannot determine the sign of the eventual corpus effect. Within the window, it identifies the direct feed effect, and an exploratory cohort analysis detects organic lift after exploration ends. The experiments establish a positive creator response, measure the gross corpus flow visible within three weeks, and show the design and duration needed to identify total value.

0.20

Beyond Raw Engagement: A Counterfactual Observability Framework for Recommender Systems at Netflix

cs.IR cs.AI ·2026-09-19 ·Chaoran Guo, Ding Tong, Ting-Po Lee, Scarlet Chen
similarity 0.61
Abstract

Understanding the performance of large-scale recommender systems remains an underexplored challenge, especially for content creators and model developers. The raw engagement signals available to them, such as views and clicks, conflate content quality, model behavior, presentation bias, and audience reach, making it hard to attribute outcomes to the right cause. In this work, we present a general evaluation framework that enhances observability across multiple recommender systems at Netflix and demonstrate its effectiveness through several production deployments. The framework treats recommender-system observability as a counterfactual measurement problem: estimating what the recommender would have done, and what engagement would have followed, in the absence of a specific content item or model decision. We articulate three stakeholder-centered observability principles for content creators and model developers, and propose measurement methodologies covering bias reduction, relativity, and incrementality, applicable to both single-stage and cascading recommender systems and serving both audiences from a single measurement foundation.

0.21

UserArena: Benchmarking Interactive User Simulation for Long-Horizon Shopping Agents

cs.IR ·2026-10-22 ·Jujia Zhao, Jiakai Tang, Jing Yu, Wen Chen, Jian Wu +4 more
benchmarkcommunity pick similarity 0.60
Abstract

Agentic recommendation pursues shopper goals through sequences of context-dependent decisions. Developing and evaluating such policies requires faithful recommendation world models that capture both the interactive shopping environment and users' behavioral responses over long interactions. Existing resources rarely provide long-horizon, fine-grained real-user trajectories or support trajectory-level evaluation and offline validation against production A/B results. We introduce UserArena, a real-world e-commerce interactive benchmark containing 64,729 session trajectories from 11,010 users, spanning 566,102 items and 13 action types. UserArena pairs these trajectories with an interactive environment and supports offline validation against outcomes from a controlled production A/B test, enabling five-level evaluation from output validity and local decisions to sequential behavior, personalization, and outcomes. To improve simulator fidelity over these long-horizon interactions, we further propose On-Policy Trajectory Distillation (OPTD), which uses trajectory-level intent to provide dense teacher targets on student-generated rollouts. Experiments show that OPTD achieves the strongest overall fidelity among the evaluated simulators under UserArena's evaluation protocol and most closely reproduces the production policy effect in offline simulation.

0.18

Amortized Off-Policy Evaluation for LLMs

cs.LG ·2026-10-07 ·Younwoo Choi, Leo Feng, Vincent Liu, Haanvid Lee
similarity 0.60
Abstract

Accurate evaluation is central to selecting which LLM to deploy, yet testing a candidate on live traffic exposes real users to an unvetted model. Teams therefore evaluate candidates offline, on data produced by already-deployed models. This is off-policy evaluation (OPE), and it faces two distribution shifts: as a model is updated in post-training, its responses diverge from the logged ones (policy shift), and the reward definition under which it is judged changes with business requirements (reward shift). Classical OPE methods are ill-suited to this continual-deployment setting because they are defined per task and require fitting from scratch on every new logged dataset or reward definition. To address this, we propose PFN-OPE, a prior-data fitted network that amortizes OPE across a distribution of contextual-bandit tasks. We pretrain it once on tasks constructed from a pool of LLM responses scored by several reward functions, in which both shifts occur. At test-time it maps a logged dataset and one sampled target response per prompt to a value estimate in a single forward pass, with no per-task fitting. On HelpSteer2 and UltraFeedback with Qwen, Llama, and Gemma policies, PFN-OPE achieves 2.0 to 9.3 times lower error than the best baselines across all tested configurations in the reward-shifted settings.

0.21