CS Paper Radar

Designing the Future of User Feedback for Generative AI

Alisa Frik, Julia Bernd, Amitis Karami, Mohammad Tahaei

cs.AI ·submitted 2026-10-02 00:39 UTC · v1
benchmark

Abstract

Post-deployment feedback from users can be a cost-effective, scalable, and representative means to monitor and improve generative AI systems and features. When implemented effectively, giving such feedback can increase users' engagement with and trust in GenAI systems. Government regulations and industry guidelines call for post-deployment user engagement, but there is little guidance on designing mechanisms that are usable for consumers and provide actionable input for product teams. We conducted a multi-phase study as a collaboration between academic researchers and eBay. Our benchmark evaluation of current industry approaches identified common issues including lack of discoverability, unclear terminology, and inattention to user value. Based on these findings, we developed best-practice recommendations and designed and tested a prototype feedback-collection tool. The tool aimed to provide users with an efficient, flexible, and positive feedback-giving experience, and provide product teams with rich data on performance and potential problems in a usable format.

Closest papers

Nearest neighbours by meaning across the whole index.

The Reported Engagement with AI Level (REAL) Rating: A Framework for Disclosing Human-AI Collaboration

cs.AI ·2026-10-02 ·Imène Goumiri, Mayleen Cortez-Rodriguez, Eric Bell, Amanda Muyskens
similarity 0.67
Abstract

Generative artificial intelligence has become increasingly incorporated into digital media and more generally into production workflows with which the public frequently interacts. Current provenance standards and disclosure methods frequently rely on binary categorizations, differentiating only between entirely human-authored and AI-generated content, or depend on technical watermarking that lacks user-facing clarity. However, there are very different risks and outcomes depending on the different uses of AI in the creation and consumption of end products. Therefore, we propose the Reported Engagement with AI Level (REAL) Rating framework which addresses these concerns by introducing a six-tier scale (Levels 0-5) that discloses the extent of AI involvement in both product creation and user experience. This paper details the structural methodology of the REAL Rating system and the specifics of its application across text, image, audio, video, and software products.

0.27

Bridging Search and CRM: Productionizing AI Product Research Agents for Customer Re-Engagement

cs.AI ·2026-08-19 ·Mandar Kulkarni, Pooja A., Samir Shah
similarity 0.63
Abstract

Modern e-commerce platforms often operate search, recommendation, personalization, and CRM systems independently, limiting opportunities for proactive customer re-engagement. This is particularly challenging for exploratory intents such as best smartphones or latest 5G phones, where users may leave the platform for external research before purchasing. We present a scalable, production-deployed framework that bridges search and CRM workflows through AI-powered Product Research Agents. The system identifies users with exploratory purchase intent and low engagement, conducts grounded multi-agent product research using behavioral signals, external knowledge, and enterprise catalog data, and delivers personalized recommendations through WhatsApp. We evaluate the framework in a 23-day production deployment involving approximately 15K WhatsApp notifications for mobile product discovery. The campaign achieved substantial CTR improvements over traditional WhatsApp recommendation campaigns, with evidence of secondary engagement through message forwarding and sharing. The deployment also generated downstream purchases and GMV impact, demonstrating the practical effectiveness of AI Product Research Agents for proactive customer re-engagement and end-to-end customer journey optimization.

0.14

Designing UIs with GenAI: Understanding the Use, Benefits, and Limitations of Prompt-to-Design Generators

cs.HC ·2026-11-09 ·Jingzhou Ye, Yao Li, Xueqiang Wang
similarity 0.62
Abstract

The integration of generative AI (GenAI) into user interface (UI) design has led to prompt-to-design generators (P2dGens), tools that enable designers to create UI prototypes from textual prompts efficiently. Despite their growing adoption, little is known about how designers use these tools in everyday practice, their benefits, and limitations. We conducted semi-structured interviews with 17 UI designers to investigate P2dGens use, focusing on workflow integration, attitudes toward generator-assisted design, and the organizational climate. Findings show that designers primarily use P2dGens as ideation accelerators rather than end-to-end solutions, and they adopt a variety of individual and collaborative prompting strategies. Designers expressed complex attitudes, noting gains in efficiency and inspiration while also highlighting concerns about loss of personal creativity and design quality. We also observed emerging governance practices and varying attitudes within organizations toward P2dGens. These insights illuminate how P2dGens reshape UI design workflows and contribute to understanding human-AI collaboration in UI design.

0.27

Player Perceptions of Generative AI in Games: A Steam Review Analysis

cs.HC ·2026-08-12 ·Mahsa Bazzaz, Seth Cooper
publishedposition similarity 0.62
Abstract

The rapid adoption of generative AI in game development has created large discussions among players, yet little empirical work has examined how players actually perceive AI-generated content. Employing quantitative methods, we study the adoption of generative AI in games in the Steam marketplace, using procedural content generation (PCG) as a baseline of a generative technology that was successfully integrated into games over several decades. Furthermore, using qualitative methods, we study player reception of generative AI by analyzing 508,192 English-language reviews. We found that games disclosing generative AI use receive lower recommendation rates and more negative overall sentiment than PCG games. Thematic analysis of 600 reviews shows that players perceive the use of generative AI in games as low developer investment in the game. Drawing on human-centered AI frameworks, we argue that successful generative AI adoption requires deploying generative AI for what players need, not for what makes development cheaper.

0.22

AutoLR: Automating the Path from Research to Launch Review in Industrial Recommender Systems

cs.AI ·2026-09-04 ·Qi Zhang, Yanlin Chen, Wenchao Xiao
similarity 0.62
Abstract

Improving an industrial recommender is an iterative research-and-engineering process rather than a direct path from idea to deployment. In \textbf{DASHEN, NetEase's gaming-community app}, algorithm engineers typically identify promising directions from research papers, technical reports, and prior production experiments; reproduce or adapt the underlying methods; implement them in the production codebase; and evaluate the resulting models through training and offline experiments. Promising candidates are then advanced to online A/B tests, and those demonstrating robust gains are submitted to Launch Review---the internal gate for full-traffic rollout. Large language models (LLMs) can assist with individual stages of this workflow, but the overall process remains human-dependent without a harness that can reliably coordinate them across long-running, often multi-day experimental cycles. We present \textbf{AutoLR}, initially built as \textbf{Auto Launch Review} and later extended upstream into an autonomous research-to-launch harness. AutoLR combines three system mechanisms: a \textbf{multi-expert council} that debates and adversarially reviews proposals; a \textbf{deterministic evidence-weighted exploration--exploitation selector} that allocates a limited trial budget across candidate directions and uses Council reranking; and a layered knowledge system that combines external research, production-system knowledge, and DASHEN-specific domain knowledge---such as game communities, player characteristics, and content-interaction patterns---with posterior evidence from configurations, patches, logs, failures, and offline outcomes. LLM agents perform semantic reasoning and code generation, while deterministic controllers retain authority over execution, metric extraction, guardrails, and persistent state transitions.

0.45

AURA: Agentic Diagnosis and Refinement for Production Recommender Systems at Scale

cs.IR cs.AIcs.LG ·2026-09-15 ·SungGeun Kim, Abhinav Narain, Daniel Nemirovsky
published similarity 0.61
Abstract · 14 pages, 1 figure, 6 tables. Accepted at GenAIECommerce'26: The Third Workshop on Agentic and Generative AI for E-Commerce, co-located with…

How and why does a recommender system fail the users it serves? Oftentimes, practitioners are left to improve their algorithms based on a combination of feedback from stakeholder teams, domain expertise, and insights from data analyses. Yet the nuances of how and where recommendations perform well or poorly for end users are difficult to discern from aggregate quantitative metrics. Whereas these metrics provide a high-level and incomplete picture, further granularity into the quality of recommendations and their patterns requires reasoning with domain understanding and objectivity, at scale. We contemplate this complex conundrum and describe a method and implementation that uses the latest AI agentic advances to provide actionable diagnoses and improvements for production recommender systems. We present AURA (Agentic Understanding and Refinement of recommender Algorithms), an end-to-end agentic system that performs qualitative evaluation at scale and can then generate improvements to our algorithms at the code level. Specialized agents read production engagement logs, from thousands of sessions to millions, and surface patterns and examples of how the recommender fails real users. The next step uses those diagnoses and context about the recommender's own code, data, and training pipeline to propose and implement refinements grounded in that codebase. We report the system design, initial tests on production data from two large consumer platforms at a major media-streaming company, safeguards, operational learnings, and early results toward a self-improving recommender system. Finally, the diagnostic gap AURA closes is not specific to streaming. The architecture is built to transfer: every domain-specific element enters through the configuration layer that already ported it between our two platforms. We map it concretely to e-commerce and online-retail recommendation.

0.41

Recursive Game Creator: An Agentic Product-Level Experience-Oriented Game Harness

cs.AI cs.MAcs.SE ·2026-10-06 ·Jiajun Chen, Haoyu Wu, Mingda Jia, Xihui Liu
code/data similarity 0.61
Abstract · Project page: https://imbaldy.github.io/recursive-game-creator/ ; Code: https://github.com/IMBALDY/RecursiveGameCreator

Recent game design agents have made substantial progress in generating playable games. However, program correctness does not ensure an enjoyable experience for players. We present Recursive Game Creator, an experience-oriented harness to advance agentic game development from rough game prototypes into entertaining games. Recursive Game Creator organizes recursive development around four components: Designer, Builder, Player, and Reviewer. The Designer translates user instructions and Reviewer's feedback into detailed plans. The Builder turns these plans into candidate games. The coding-native Player creates and executes reusable policies through programmatic interfaces to efficiently collect diverse gameplay trajectories, mitigating evaluation bias caused by slow GUI-based collection. The Reviewer uses carefully designed trajectory-based metrics to induce player preferences, integrating with visual evidence and explicit textual preferences to evaluate games against game-specific criteria. Finally, the Reviewer accepts the better version and provides improvement reviews for the next round, closing the recursive loop. Our method achieves state-of-the-art overall performance of 77.89 on GameCraft-Bench. On GameASG-Bench, it achieves a strict task success rate of 53.2%, a 34.1% improvement over the same-model baseline, and the highest mean runtime-check pass rate at 93.4% among compared methods. A user study shows longer playtime and higher ratings. Project page is at: https://imbaldy.github.io/recursive-game-creator/.

0.53

TractorBeam: Personalized AI Sensemaking Support via Collaborative Machine Annotation

cs.HC ·2026-08-19 ·Sireesh Gururaja, Jordan Taylor, Emma Strubell
published similarity 0.60
Abstract · UIST Poster Extended Abstract

Language model-based systems which allow asking questions of documents have become popular tools for sensemaking. Despite their implied capability, these systems still suffer from issues of factuality and provenance, while encouraging confirmatory, rather than exploratory, research. We present TractorBeam, a browser extension-based mixed-initiative system that uses collaborative annotation as an interface metaphor for sensemaking, re-framing language model (LM) outputs as suggested highlights in a process that we call collaborative machine annotation. This metaphor allows us to present LM results in-context on PDF documents, directly addressing concerns of provenance and factuality, while allowing users to iteratively construct mental schemas and queries for language models directly in the context of a document. In a preliminary user study, all of our participants felt that TractorBeam enabled them evaluate and iteratively improve the model's reflection of their intended highlighting, and several found suggestions that made them reconsider their original schema. TractorBeam suggests that systems that facilitate exploratory research on individual documents may lead to verifiable sensemaking for users and complement tools that work across broader corpora.

0.22