CS Paper Radar

Harnessing Generative UI for Education: Tailored Learning Interactives

Alisa Kovshov, Anisha Choudhury, Anna Iurchenko, Amy Keeling, Avinatan Hassidim, Ayelet Shasha Evron, Ayça Çakmakli, Diana Akrong, Femi Olanubi, Gal Elidan, Ian Li, Ido Lerer, Katherine Chou, Lidan Hackmon, Michal Gordon, Nir Kerem, Niv Efron, Preeti Singh, Rena Levitt, Rotem Yulzary, Shlomi Ben Shimon, Sophie Allweis, Tracey Lee-Joe, Tzvika Stein, Yaniv Carmel, Yael Haramaty, Yishay Mor, Yossi Matias, Yuri Lev

cs.CY ·submitted 2026-09-17 17:26 UTC · v1

Abstract

It is well established that students learn best through active engagement, particularly via interactive experiences tied to real-world concepts. Creating such experiences, however, typically requires labor-intensive design, costly development, and continuous teacher guidance to achieve measurably effective outcomes. Recent advances in generative UI open the door for a future of practice where the creation of tailored interactive experiences is automated. However, off-the-shelf models lack optimization for pedagogical precision and learning principles, often yielding unsatisfactory results for complex pedagogical constructs. To address these limitations, we present learning interactives, a structured framework leveraging the generative UI capabilities of Gemini to construct effective customized interactive simulations. We evaluate our approach through both an initial study with educators as well as a pedagogical rating of a varied set of simulations by expert teachers.

Closest papers

Nearest neighbours by meaning across the whole index.

Primer: An AI Framework for Instructor-Centered Educational Simulation Creation

cs.AI ·2026-11-12 ·Ethan Mollick, Lilach Mollick, Ben Przystanski, LJ Ciccarelli, Natalie Bach +1 more
code/data similarity 0.68
Abstract

Educational simulations are a well-established pedagogical technique that gives learners opportunities to apply knowledge, respond to challenges, and reflect on their performance. High-quality simulations, however, are difficult to build, requiring instructor input, scenario design, character development, scaffolding, technical implementation, and repeated revision. This paper presents Primer, an open-source, LLM-agnostic AI-supported system that helps educators create adaptive educational simulations. Educators contribute their knowledge of the subject, lesson, and students, while generative AI produces responsive variations of practice for students. Primer uses a multi-agent system centered around executing educator's instructional intent through the entire learning experience. We hope that by releasing Primer as an open-source system, including its prompts and GitHub repository, we provide a foundation for development and research on educator-in-the-loop AI authoring, multi-agent educational workflows, and simulation-based pedagogy. Further, by allowing the use of various AI models, including local and open source models, we provide the opportunity for implementation under varied governance regimes. Primer points toward a model of educational AI in which instructors can create locally grounded practice opportunities based on their own materials, goals, examples, and knowledge of students.

0.30

Educating future engineers about LLMs: A scalable workshop

cs.HC cs.CYcs.RO ·2026-10-04 ·R. Zhang, J. C. F. de Winter, T. Dicke, D. Dodou, Y. B. Eisma
code/data similarity 0.61
Abstract

As large language models (LLMs) are increasingly integrated into engineering workflows, students require hands-on experience to learn how to collaborate with them critically. This paper presents a scalable gamified workshop designed for engineering Master's students to practice human-AI collaboration in navigation planning. Using a mobile web interface across 10 workshop sessions, a total of 226 students wrote prompts for a non-reasoning and a reasoning LLM to solve grid-based navigation tasks of increasing complexity. The system returned robot-executable plans, trajectory visualizations, and automated scoring, culminating in a live demonstration on a Boston Dynamics Spot robot. In a post-workshop questionnaire, 81.5% reported substantial learning and 91.0% reported high engagement. Analysis of the submitted prompts revealed that students changed their strategies from step-by-step instructions for the non-reasoning LLM toward providing higher-level guidance for complex problem-solving tasks. We conclude that such interactive simulation-to-reality environments are viable for teaching the verification and collaboration skills necessary for responsible LLM use in engineering. Code is available at: https://github.com/renchizhhhh/LLM-robotics-workshop

0.38

INSIDE the Student's Mind: Jointly Modeling Latent Reasoning and Action in LLM Student Simulators

cs.AI cs.CY ·2026-08-11 ·Rose Niousha, Minwoo Kang, Narges Norouzi
published similarity 0.60
Abstract · Accepted at the Conference on Language Modeling (COLM) 2026

Large Language Model (LLM)-based simulators often reproduce observable actions but fail to capture the underlying reasoning behind them. In education, where student simulation is increasingly used for various applications such as evaluating tutoring systems, this gap is especially pronounced. Two students may submit identical submissions for entirely different reasons. We present INTERNAL STUDENT DIALOGUE (INSIDE), a student modeling framework that fine-tunes LLMs not only to act like students but also to think like them. INSIDE generates internal dialogue grounded in Bloom's Taxonomy across cognitive, affective, and action dimensions, and fine-tunes models on paired think traces and actions. We baseline against different prompting frameworks and evaluate on two axes: fidelity of simulated actions and quality of generated internal dialogue. Our evaluations show that INSIDE improves simulation fidelity in both action fidelity, matching code generation of real students, and reasoning alignment, achieving the highest alignment across models up to 57.9%.

0.49

Integrating Slides, Mathematics, and Simulation in One Page: GenAI-Assisted HTML Learning Objects for Engineering Courses

cs.HC ·2026-11-11 ·Xuan-Quy Dao, Ngoc-Bich Le, Xuan-Dung Phan, Le-Minh Nguyen
similarity 0.60
Abstract · 25 pages, 5 figures, 11 tables

Engineering courses spread their materials across slides (PPTX), typeset notes (LaTeX/PDF), and MATLAB, leaving students to connect them. We describe an instructor-in-the-loop workflow in which generative AI (GenAI) converts these assets into single-file HTML learning objects that combine presentation, mathematics, visualization, simulation, and assessment; a page is released only after its simulations pass numerical verification. Six open demonstrators implement 18 tasks from six engineering courses. In the eight tasks with quantitative cross-checks, every quantity met its tolerance at default settings (1\%, or 2\% for the finite-difference time-domain simulation) and agreed with a separate Python re-computation and a value-blinded MATLAB re-computation run by a colleague not involved in development; the checks also flagged under-resolved settings. In a perception survey, students (19 after post-hoc screening of 28 responses) reported that the pages linked equations with simulation, and 74\% preferred them to separate files, but usability was marginal (System Usability Scale 56.8). Twelve lecturers rated the approach as useful (item means 4.25--4.58) but were hesitant to adopt it; their median ratings on a 12-criterion capability rubric agreed moderately with the first author's ($κ_w=0.60$; individual median 0.31). The results show technical feasibility and preliminary positive perceptions; learning outcomes were not measured.

0.47

A Design Space for Visual Interfaces for Generative Image Models

cs.HC ·2026-09-16 ·Susie S. Y. Li, Mingwei S. G. Li, Remco Chang
similarity 0.60
Abstract

Interactive visual interfaces have become an important means of controlling generative image models, enabling users to manipulate generation through prompts, direct manipulation, and a range of interactions. However, existing techniques are typically presented as independent systems, making it difficult to understand how they relate, compare their interaction mechanisms, or identify opportunities for new interface designs. We introduce a design space for interactive visual interfaces for generative image models derived from 51 research systems and practitioner tools. The framework decomposes each system into three complementary components: the user interface (U), the controllable model objects (Z), and the mapping function ($φ$) that translates user interaction into model operations. This decomposition provides a common representation for analyzing heterogeneous interaction techniques across model families, revealing recurring design patterns and underexplored regions of the design space. We further present an interactive corpus explorer that support comparative analysis, and discuss usage scenarios for both educational settings and HCI/AI practitioners identifying research and design opportunities.

0.07

From Content Generation to Learning Support: Pedagogy-Guided Generative Video Tutors for STEM Learning

cs.CL cs.AIcs.CY ·2026-09-21 ·Xinchen Ma, Shuimu Wang, Gaole He, Yanbin Zhang, Chunyang Wang +2 more
top venuecode/data similarity 0.60
Abstract · accepted to EMNLP 2026, code available at GitHub

Generative AI enables scalable production of educational videos, but current systems largely focus on producing visually coherent content rather than supporting learning. As a result, generated videos often lack explicit pedagogical structure, reliable quality control, and mechanisms for assessing learner understanding or addressing misconceptions. In this work, we introduce PIVOT (Pedagogy-guided Instructional VideO Tutoring), a generative video tutoring framework for STEM learning via learning-centered instructional support.1 Inspired by conventional teaching workflows, our framework integrates pedagogy into the full generation pipeline: it first uses instructional principles to guide storyboard generation, then produces verified multimodal videos through code-centric generation and a pedagogical verification harness, and finally connects videos with assessment and misconception-aware remediation. Experiments and expert evaluations across four STEM domains show that our framework produces educational videos with pedagogically aligned content, clear and engaging presentation, coherent instructional flow, and perceived effectiveness for learning. These findings suggest a human-centered perspective on educational content generation: generative systems should be evaluated and designed not only by what they produce, but also by how they support teaching practices, learner understanding, and corrective feedback.

0.54

LumiNote: LLM-Assisted Multimodal Instruction for VR Stage Lighting Education

cs.HC ·2026-09-15 ·Danxuan Liang, Chun Yin Li, Zheng Wei, Xian Xu, Meng Xia +2 more
similarity 0.60
Abstract

Stage lighting education requires instructors to bridge abstract concepts, technical operations, and learner-understandable representations. While Virtual Reality (VR) removes physical constraints, existing systems provide limited support for live instruction. We present LumiNote, an LLM-assisted VR system that transforms spoken pedagogical intent into instructor-reviewable spatial annotations, executable demonstrations, and linguistic support. In an exploratory study with 3 instructors and 24 students, we examined how instructors incorporated LumiNote into familiar lighting topics and how students received the resulting representations. We found LLM assistance most valuable for expressive, under-specified goals, but requiring greater expert intervention for fixture-specific or spatial configuration requests. Instructors engaged with generated suggestions as a controllable refinement process, shifting effort from manual setup toward pedagogical expression. However, representations that externalized expert reasoning did not always align with novice comprehension. These findings characterize LLM-assisted VR instruction as a domain-grounded mediation process among expert expression, executable operations, and learner-facing representations.

0.24

"Act Like a 5th Grader" is Not Enough: Bounding Knowledge in LLM-Based User Simulators

cs.CL cs.AI ·2026-08-30 ·Krisztian Balog, Arild Michel Bakken
top venue similarity 0.59
Abstract · Findings of the Association for Computational Linguistics: EMNLP 2026

Large language models (LLMs) are increasingly used to simulate human behavior but frequently fail to exhibit realistic cognitive constraints, suffering from a "superhuman bias." Using a dataset of over 71,000 reading comprehension responses from 2,359 primary-school students (grades 4--6), we demonstrate that standard persona prompting yields near-perfect, deterministic performance, failing to capture the natural variance of developing readers. To address this, we introduce the Cognitively Bounded User Simulator (CBUS), an architectural framework that explicitly models the restricted working memory of young readers through an episodic bottleneck. Within this framework, we formalize two distinct test-taking strategies to emulate different reading behaviors. Our evaluation shows that explicitly modeling cognitive bounds significantly narrows the simulation gap across multiple LLM backbones, demonstrating that enforcing architectural constraints is more effective for high-fidelity simulation than simply scaling raw model capabilities.

0.50