CS Paper Radar

Towards Audience-Driven Real-Time Data Storytelling

Angelica Lo Duca

cs.HC ·submitted 2026-10-29 09:12 UTC · v1
published

Abstract

This paper introduces the concept of Audience-Driven Real-Time Data Storytelling (ADRDS), a framework that utilizes participatory and social sensing to enable audiences to actively shape the presentation of data stories. At its core, ADRDS employs a pre-authored decision tree, where the presenter defines the data story through a branching three-act arc (setup, conflict, and resolution). Each node contains a question, branching rules, thresholds for coverage, consensus, and confidence intervals, as well as one or more potential calls to action (CTAs). Audience members connect to the ADRDS framework via a URL on their mobile devices and answer questions in real time. ADRDS evaluates responses against the decision tree in real-time, following strong branches when thresholds are satisfied or activating fallback paths when signals are inconclusive. ADRDS then selects the final action consistent with the traversed path and the audience's demonstrated readiness, subject to fairness safeguards. ADRDS transforms presentations of data stories into audience-shaped decision processes, where live data actively drives the narrative. This work paves the way towards a new form of data storytelling, where the final data story depends on the audience's background and interests.

Journal ref
CHIRA 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Live Assistant: Learning Whether, When, and Whom to Assist in Real-World Live Social Streams

cs.LG ·2026-09-23 ·Shujian Gao, Jiamei Yan, Yuchen Yang, Penghao Zhou, Qinglei Wang +4 more
benchmark similarity 0.58
Abstract · under review

Livestreams are long-lasting interactive environments where audiovisual content, viewer activity, host behavior, and platform signals evolve together, creating assistance needs that emerge from the stream itself. We introduce \liveassistant, a framework for mixed-initiative, role-conditioned assistance that formulates livestream interaction as four coupled decisions: \textbf{whether to act, when to act, whom to address, and what to communicate}. At each 10-second interval, one autoregressive policy consumes native audio and video with synchronized comments, gifts, viewer dynamics, and room metadata, then selects \textsc{OBS}, \textsc{MEM}, or \textsc{ANS}. \textsc{OBS} remains silent, \textsc{MEM} records a private semantic update, and \textsc{ANS} specifies a recipient, task, and grounded message. To support this task, we build a trajectory engine that reconstructs real livestream sessions into structured causal supervision, yielding over 320 hours of optimization trajectories and a human-reviewed benchmark of 275 clips and 13,812 decision intervals. We train the policy with Marker-Aware Multiturn Supervised Fine-Tuning (MA-MSFT), which strengthens sparse structured decisions, followed by Streaming Multiturn GSPO (SM-GSPO), which optimizes self-generated trajectories with turn- and trajectory-level credit. On the held-out benchmark, \liveassistant reaches 71.14 state accuracy, 72.67 recipient accuracy, and 58.41 task accuracy, with consistent gains over representative streaming and general multimodal baselines. Together, the formulation, benchmark, and training framework establish livestream assistance as selective participation in a shared social stream.

0.18

Semantic Action Graph: A Shared Representation for Agent Grounding and Human Interpretation of Sports Highlights

cs.HC cs.AI ·2026-09-17 ·Tica Lin, Deepak Chandran, Gauri Jagatap, Chen Chen, Andrea Fanelli +2 more
similarity 0.56
Abstract · 5 pages, 3 figures, Accepted for publication at IEEE VIS 2026 Workshop on GenAI, Agents, and the Future of VIS

Generative agents are increasingly used to select and narrate video highlights, but they typically operate over unstructured or frame-level representations. Their output is consequently difficult for a viewer to verify and steer toward individual preferences. We present the semantic action graph, a lightweight domain schema that represents a sports match as performer, action, recipient, moment, and state nodes connected by role, temporal, and outcome edges. The schema demonstrates three key properties: 1) connected event sequences, 2) a shared, closed vocabulary, and 3) frame-addressable moments, making it suitable to serve two consumers at once: an agentic pipeline that composes narrated highlights, and a visual interface through which viewers query and inspect the same structure. We instantiate it in SportSAGE, a design probe pairing a four-module highlight pipeline with a graph interface, and report feedback from 12 soccer fans. Participants were satisfied with the quality of the generated highlights and narratives, and used the graph interface to search, navigate, and interpret the match highlights. These results provide early evidence that one small, human-readable schema can ground agent generation and support human interpretation at the same time.

0.13

Realtime-Bench: Can Frontier Models Solve Real-time Interactive Problems?

cs.AI ·2026-10-15 ·Jingyu Tang, Xu Yang, Zhehao Zhang, Zheng Wang, Qi Cheng +5 more
highly cited similarity 0.55
Abstract

Real-time interaction requires agents to perceive evolving environments and act under temporal constraints, yet existing evaluations of real-time streaming models focus on conversational quality and streaming perception, leaving effective action under strict temporal constraints untested. We introduce Realtime-Bench, a suite of carefully curated dynamic tasks where delayed actions directly affect success. Under Wall Clock, where environments advance according to actual inference latency, the best success rate reaches only 44% and the best Pass^3 is only 36%, demonstrating that real-time interaction remains challenging. To separate serving latency from models' real-time streaming capability, we compare Wall Clock with Paused Clock, which removes inference delays as an idealized reference, and introduce Token Clock, which advances simulated time according to generated token counts, testing temporal awareness independently of physical serving speed. We find that temporal costs limit success in evolving environments and that higher reasoning effort has mixed effects under time pressure, reflecting a trade-off between decision quality and timely execution. Based on these results, we discuss future directions for improving real-time multimodal models, including adaptive reasoning with temporal awareness and continuous perception and action.

0.34

Graphing the Everyday: A Neurosymbolic Approach to Eliciting Routines for Just-In-Time Adaptive Interventions

cs.HC ·2026-08-10 ·Shakyani Jayasiriwardene, Blake Mountford, Meican Ma, Niels van Berkel, Nicholas Koemel +4 more
published similarity 0.54
Abstract · Accepted at OzCHI 2026

Just-In-Time Adaptive Interventions (JITAIs) increasingly rely on conversational agents to elicit user routines, yet translating fluid human dialogue into rigid schedule data remains a significant challenge. We conducted a qualitative investigation of a neurosymbolic pipeline, combining Large Language Models (LLMs) with a Neo4j knowledge graph, to map unstructured verbal narratives into actionable interventions. Through human-centric evaluation using natural-language playbacks, we identified a critical "mental-model gap," where the linear extraction of LLMs clashes with hierarchical, non-linear human storytelling, causing severe entity fragmentation. Furthermore, we articulate an "ecological mismatch," demonstrating that algorithmic schedule availability frequently ignores the user's fluctuating psychological receptivity and physical energy levels. To resolve these tensions, we propose actionable design heuristics, including routine piggybacking, adaptive negotiation, and scalable transparency. Ultimately, these guidelines provide a foundational framework for evolving rigid schedule-trackers into empathetic, context-aware proactive agents capable of supporting long-term health behavior change.

0.19

Balancing Evidence and Interpretation: Historical Grounding Ratio as a Design Parameter for AI-Generated Urban Storytelling

cs.HC ·2026-08-25 ·Fuyang Zhang, Maurice Benayoun
hot direction similarity 0.54
Abstract · 34 pages, 11 figures, 4 tables

Location-aware generative systems can now select historical archives and real-time contextual information based on a user's surroundings to automatically generate narratives for urban heritage walks. Yet when multiple sources jointly inform generation, existing systems provide neither a clear representation of how much content from each source actually appears in the output nor an operational means of measuring it. We introduce the Historical Grounding Ratio (HGR), defined as the proportion of claim-bearing information units in a generated narrative that are supported by historical archives. HGR turns the realized share of historical evidence in a narrative into a directly measurable design parameter. In GeoDrama, a mobile narrative system, we created three conditions that used a common retrieval procedure and comparable evidence-bundle sizes while varying the allocation of information from different sources during generation. We evaluated how changes in HGR affected narrative experience through a within-subject walking study with 18 participants. Increasing HGR significantly strengthened the perceived relevance between narrative content and the specific location. However, historical understanding, integration with the visible scene, appropriateness of the amount of information, and intention to explore further did not increase monotonically with HGR; all four measures were highest in the intermediate, balanced condition. These findings show that designing location-aware generative interfaces involves not only retrieving relevant material but also determining how information from different sources composes the final output. HGR offers an operational measure for comparing information-allocation strategies and their experiential consequences.

0.38

AskStream: When and What to Ask Proactively under Uncertainty in Streaming Video Understanding

cs.CV cs.CL ·2026-10-28 ·Lu Wang, Zhuoran Jin, Longxiang Wang, Junhong Chen, Yupu Hao +6 more
benchmarkcode/data similarity 0.54
Abstract · 42 pages, 9 figures. Project: https://www.askstream.top ; Code: https://github.com/wanglu-cs/AskStream ; Dataset: https://huggingface.co/dat…

Recent advances in multimodal large language models have increased interest in interaction paradigms for video understanding. The evolution from offline to streaming and proactive interaction has gradually shifted models from passive responders toward more proactive agents that can decide when to answer. However, existing proactive paradigms typically assume that the information required to answer a query is complete, leaving models to only wait or answer. In the real world, however, information is often incomplete due to uncertainty from either the video or the user. In proactive paradigms, such uncertainty can cause the model to wait forever or generate highly hallucinatory responses. We therefore argue that proactive interaction should be redefined with a third action: ask, allowing models to actively ask for missing information. To study when and what models should ask before answering, we introduce AskStream, the first multi-turn benchmark spanning video and user uncertainty for streaming video understanding, with four task categories and eight subtasks. It contains 438 videos with 2,714 manually annotated frame-level time windows across three types: wait, ask, and answer. Across both multiple-choice and open-ended settings, even the strongest model attains only 52.18% on our proposed Ask-F1 metric, highlighting substantial limitations in proactive information seeking under uncertainty.

0.38

The Garden of Forking Prompts: How Users Explore Narrative Space in Story Generation

cs.CL ·2026-09-13 ·Advait Deshmukh, Nora Benedict, Melanie Walsh, Maria Antoniak
published similarity 0.53
Abstract · Accepted at COLM 2026

Large language models (LLMs) have changed the way people engage with stories. Drawing on public chatbot logs, we can see that when users generate stories, they iteratively edit their prompts to explore narrative possibilities, adjusting characters, redirecting plots, and swapping fictional universes. As aggregated data, these prompts represent rich traces of creative preference at scale. Yet story generation evaluation benchmarks rely on static, one-shot prompts that cannot capture this exploratory behavior. In this work, we study how users revise consecutive story prompts in the wild. Using a dataset of naturally occurring user-chatbot conversations, we construct WildStories, a sample of 275,635 story generation prompts (labeled with story format, prompt components, and explicitness), and WildEdits, a collection of 24,291 edit trees that model how users iteratively edit base story prompts and explore branching story possibilities. From these trees we develop a framework of edit types crossing four directions (adding, removing, changing, and extending) with fourteen targets (e.g., plot, character, genre). We then use our datasets and this framework to analyze user behavior in navigating narrative space via LLMs. Finally, we show how automated permutations based on the framework can be used for story generation benchmarking. Content Warning: This paper works with "wild" chatbot logs, which often include toxic and sexually explicit themes.

0.32

Why This and Not That? A Collaborative Reflection Approach for Understanding Thought Coverage in Decision Making Support Dialog

cs.HC ·2026-08-17 ·Morita Tarvirdians, Hayley Hung, Catharine Oertel
top venue similarity 0.53
Abstract · Accepted to Findings of the Association for Computational Linguistics: EMNLP 2026

Conversational agents that support reflection for decision-making often rely on adaptive dialog policies that map observed user behavior to actions such as probing, deepening, or redirecting. Yet the same pattern can reflect a range of different reasons such as deliberate prioritisation or limited self-access. By modeling the observable pattern rather than the user's reason for it, current policies risk premature assumptions about the user state and inappropriate next actions. To address this gap, we introduce a human-centered method for surfacing this hidden inference step. In a user study with 62 users and 232 collaborative moments, we pause a reflection-support agent when it would normally redirect the conversation, surface its observation, and ask users to interpret the pattern and decide how to proceed. We derive a taxonomy of nine interpretation categories and show that similar reflective states can call for substantially different follow-up actions. Our findings challenge the assumption that adaptive dialog policies can rely on observable behavior alone, and suggest how user-provided interpretations can inform more appropriate conversational actions.

0.46