CS Paper Radar

ClickSight: An LLM Framework for Interpreting Student Clickstreams Through Learning Strategies

Bahar Radmehr, Lan Tran, Ekaterina Shved, Fatma Bet{ü}l G{ü}reş, Adish Singla, Tanja K{ä}ser

cs.AI cs.CLcs.CY ·submitted 2026-11-01 16:09 UTC · v1
code/data

Abstract

Digital learning environments produce rich clickstream data that can reveal how students approach learning tasks, yet interpreting these fine-grained traces often requires substantial expert effort. We introduce ClickSight, a framework for interpreting student clickstreams through learning strategies defined in the literature. Given a student clickstream and a predefined set of learning strategies, ClickSight uses a staged LLM-based pipeline to identify strategy-relevant evidence, assess adherence to each strategy, and generate an evidence-grounded interpretation of the student's behavior. We evaluate ClickSight across two learning environments, 365 students, and 12 model backbones using an expert-grounded quality rubric and an LLM judge calibrated with human-expert ratings. Our results show that ClickSight can generate high-quality strategy-based interpretations across different models and environments, with reasoning models performing more consistently than instruction-tuned counterparts. We further find that providing external clickstream-extracted guidance can improve interpretation quality for strategies that require precise quantitative information from the trace. By connecting fine-grained clickstreams with theoretically defined learning strategies and preserving the evidence underlying each judgment, ClickSight provides a flexible approach for studying student behavior in digital learning environments.

Comment
29 pages, 12 figures, 19 tables. Under review. Code: https://github.com/epfl-ml4ed/ClickSight-CEAI

Closest papers

Nearest neighbours by meaning across the whole index.

Modeling Student Sensemaking with LLMs and Knowledge-Graph-Guided Inference

cs.CL ·2026-09-25 ·Özge Alacam, Zübeyde Demet Kirbulut Güneş, Funda Ekici, Nurcan Turan-Oluk, Dilay Dinçdemir +5 more
similarity 0.64
Abstract

Collaborative science learning requires nuanced interpretation of student dialogue to characterize how learners identify knowledge gaps, build explanations, and work toward resolution - a theory-driven analysis that is labor-intensive and difficult to scale. We investigate whether instruction-tuned large language models (LLMs) can support multidimensional analysis of collaborative sensemaking without task-specific training, and whether structured knowledge-state information improves model inference. We evaluate two mid-size LLMs on 23 richly annotated, expert-labeled episodes across prompting conditions that vary definitional scaffolding, reasoning mode, and turn structure. Without reasoning, models tend to overpredict successful sensemaking; reasoning-enabled prompting improves identification of unsuccessful cases. Knowledge-state diagnostics provide additional grounding, improving detection of unsuccessful sensemaking and increasing agreement with expert annotations. No single configuration performs best across all sensemaking dimensions, underscoring the multidimensional nature of the task.

0.18

MAVE: A Multi-Agent Framework for Grounded Pedagogical Feedback from Online Class Videos

cs.CY ·2026-10-17 ·Qingyue Chen, Huaiyuan Yao, Wanpeng Xu, Varun Shourie, Qianwen Wang +1 more
benchmarkcode/data similarity 0.64
Abstract · 18 pages. Code available at https://github.com/DaRL-GenAI/MAVE

Timely pedagogical feedback is difficult to scale in online classes, where long and noisy videos make student responses hard to locate and automated systems often produce generic advice weakly grounded in actual classroom behavior. The key challenge is to connect the instructional goal of a specific lesson segment with temporally aligned multimodal evidence of student responses, and then translate the observed gap into an actionable teaching move. We introduce MAVE (Multi-Agents for Video-based Educational Feedback), a multi-agent framework for grounded pedagogical feedback from online-class videos. MAVE decomposes feedback generation into four roles: the Curator structures the lesson and infers expected student activity; the Observer extracts aligned speech, visual, and interaction evidence; the Assessor estimates the activity supported by student behavior; and the Advisor generates gap-conditioned teaching recommendations. To evaluate this capability, we introduce MAVEBench, a benchmark designed to measure whether feedback systems can ground diagnoses in multimodal student evidence, and generate actionable pedagogical feedback. MAVEBench contains 173 knowledge-point segments from approximately 20 hours of authentic online-class videos across STEM and Humanities, together with comprehensive simulation. Experiments show that MAVE outperforms direct prompting, Reflexion, and TextGrad in diagnostic correctness, instructional alignment, and actionability. Our code is available at https://github.com/DaRL-GenAI/MAVE

0.36

Mitigating AI Risks in Computing Education via LLM-Driven Lecture Video Curation

cs.CY ·2026-08-17 ·Owen Tang, Alexandra Vassar, Jake Renzella
similarity 0.63
Abstract · 7 pages, 3 tables, 1 figure

This study evaluates the effectiveness of utilising large language models (LLMs) to retrieve targeted segments from delivered video recordings to answer student questions in introductory programming environments. By restricting AI to identifying existing, educator-verified media rather than generating open-ended text, this approach aims to mitigate common pedagogical risks such as generative hallucinations and cognitive bypassing. We benchmarked three distinct models, two proprietary (Gemini 3.1 Pro and GPT 5.4 Pro) and one open-weight (Qwen3.5 397B), against a human lecturer's manual video selections. An automated judging framework subsequently assessed the outputs for relevance, sufficiency, redundancy, and the presence of extraneous material. While the AI-retrieved timestamps rarely shared exact overlaps with the human baseline, the proprietary models achieved near-parity with the expert in delivering sufficient and highly relevant answers. Furthermore, a pilot deployment of this retrieval system in a large C programming cohort (n~=900) demonstrated strong user engagement, with students primarily utilising the tool to review foundational concepts. By leveraging AI to retrieve established lecture material, this approach shows potential for a reliable, high-fidelity pathway for safely integrating LLMs into novice computing courses.

0.12

TutorLoop: Regulating Student Learning Behaviors via Sensor-in-the-Loop Generative Feedback

cs.HC cs.AI ·2026-10-07 ·Songlin Xu, Xinyu Zhang
similarity 0.63
Abstract

We present TutorLoop, a sensor-in-the-loop system that regulates student learning behaviors by delivering adaptive feedback based on real-time cognitive states. Unlike prior large language model (LLM) tutors that directly depend on scenario-specific content, TutorLoop operates on sensor-derived signals captured via webcams. Moreover, unlike direct cognitive-to-feedback mappings that are short-sighted, the system employs a deep reinforcement learning (DRL) agent to optimize the feedback type across the entire learning process. Finally, another LLM tutor refines feedback into human-like, context-aware messages. We evaluate TutorLoop in a large-scale user study (N=187), where a model trained offline is directly applied to a new learning task without retraining. Results show that TutorLoop provides less frequent yet more effective interventions, improving attention, reducing workload, increasing engagement, and ultimately enhancing learning outcomes. These findings highlight the potential of closed-loop, sensor-driven feedback for scalable human-AI integrated systems to support learning.

0.22

Video-MOPD: Multi-Teacher On-Policy Distillation for Video Understanding

cs.CV ·2026-09-08 ·Zhenxin Qin, Peng Shi, Cong Han, Yinlong Qian, Zequn Jie +1 more
code/data similarity 0.62
Abstract · Technical report

Video understanding demands a convergence of complementary capabilities across perception, temporal understanding, and complex reasoning, which are difficult to jointly optimize within a single model. We introduce Video-MOPD-8B, an open-weight model dedicated to video understanding tasks. To fundamentally enhance its capabilities, we conduct targeted reinforcement learning (RL) optimization across three core domains: video temporal grounding (VTG), general video comprehension, and video STEM reasoning. We then unify their complementary capabilities via Multi-Teacher On-Policy Distillation (MOPD), which consolidates expert knowledge by supervising student-generated trajectories with routed teacher feedback. We further introduce Reliability-Aware Informative Sampling (RAIS), which selects examples with consistently reliable teacher supervision and large teacher-student performance gaps. Together, these components enable Video-MOPD-8B to achieve coordinated and comprehensive performance gains across diverse video understanding tasks. Extensive experiments on comprehensive benchmarks covering general video understanding, temporal grounding, video reasoning, and video STEM tasks demonstrate that Video-MOPD-8B achieves state-of-the-art performance among existing models at a comparable scale. The trained model weights are available at https://huggingface.co/LandH/Video-MOPD-8B.

0.43

What Do Digital Traces Predict? Task Effects, Student Differences, and Critical Online Reasoning

cs.CL cs.CYcs.LG ·2026-11-16 ·Maxim Konca, Alexander Mehler
similarity 0.61
Abstract

Digital assessment traces are increasingly used to predict student performance, often evaluated by correlating predicted and observed scores pooled across tasks. However, such correlations may reflect between-task differences rather than performance differences among students within tasks. To investigate this, we analyzed up to 527 browsing sessions from 424 university students completing open-Internet Critical Online Reasoning tasks and assessed the cross-task stability of their ratings. We evaluated the same out-of-fold predictions within and across tasks, compared three validation tiers up to leave-one-task-out, and tested incremental predictive contributions of consultation order and relational structure against matched permutation and randomization controls. None of the twelve prespecified primary tests was significant: on held-out tasks, consulted-page features showed no significant association with relative student performance, nor did observed order or relational structure add predictive value. Pooling observations across tasks substantially increased correlations, particularly for factual correctness, whose scores showed limited evidence of stable between-student differences across tasks. However, sentence embeddings yielded modest positive correlations with generic response quality on held-out tasks, without establishing added predictive value from observed order or relational structure. Our findings show that pooled predictive correlations may not reflect within-task student differences, highlighting the need for within-task evaluation and criterion generalizability audits.

0.37

Edhivo: An AI-Native Platform for Scalable Assessment and Instant Feedback in Higher Education

cs.CY ·2026-10-27 ·Moraish Kapoor, Jiawei Zhou
similarity 0.61
Abstract

Scaling assessment in higher education faces two intertwined bottlenecks: the cognitive burden of authoring assessments and the inability to semantically evaluate open-ended responses, which together create a persistent feedback gap that hinders student learning outcomes. To address these challenges, we present Edhivo, an AI-native pedagogical platform that addresses both bottlenecks through a vertically integrated system for content delivery, assessment, and reporting. Edhivo employs a Dual-LLM pipeline: one model generates context-aware assessments for both multiple-choice and open-ended questions, from instructor-provided materials, while a second model performs multi-criteria, rubric-based evaluation of student responses, enabling near-instant qualitative feedback at multiple hierarchical levels. All AI interactions are source-grounded to reduce hallucination risk, and a Human-in-the-Loop review process preserves instructor authority. We demonstrate Edhivo's efficacy through a pilot deployment at Stony Brook University in a real classroom environment in 2026 spring, where it supports high-frequency assessment creation and evaluation for a cohort of 36 students throughout a full semester, significantly reducing authoring effort and feedback latency while enabling novel pedagogical designs centered on active learning.

0.17

MisconceptDialogue: Evaluation Criteria Shape Measured Success in LLM Math Tutoring

cs.CL ·2026-10-01 ·Woojin Lee, Yoonkyu Woo, Jinah Kim, Hyeok-Min Gwon, Jin-Xia Huang
publishedbenchmark similarity 0.61
Abstract · Accepted to the AACL-IJCNLP 2026 Main Conference. 20 pages, 1 figure, 16 tables

Large language models are increasingly used as math tutors, raising two questions: can a single LLM detect a student's error, diagnose the underlying misconception, and remediate it through multi-turn dialogue - and can we trust the numbers that report how well it does? We address both with MISCONCEPTDIALOGUE, a fully automated framework that runs the full pipeline on five open-weight LLM tutors (7B-14B) interacting with LLM-simulated students over 5,000+ dialogues - a deliberately simulated protocol (paired conditions, a wrong-diagnosis control unsafe with real learners, scale) whose results do not establish real-student learning. On measurement, our headline finding is uncomfortable for the field: benchmark-style exact-match scoring common in standard math NLP - a lower-bound stress test - reports 42.1% measured Remediation Success Rate on identical dialogues where a human-calibrated LLM-as-judge reports 81.3%, a 39.2 percentage-point spread larger than the combined effects of tutor model, teaching strategy, and supplied diagnostic information. This is not a single-judge artifact: four independent LLM judges and 200 human annotations converge to within a few points (kappa up to 0.91). On capability, the verdict is equally striking: the score barely changes whether we tell the tutor the student's actual misconception or a deliberately swapped one - under the tested diagnosis-presentation format, these tutors show little evidence of reliably conditioning on the diagnostic information they are handed. For LLM tutoring evaluation to mature, multi-criterion scoring with multi-judge convergence and human anchoring should become standard practice - and the measurement question deserves as much scrutiny as the capability question it claims to answer.

0.32