CS Paper Radar

Perspicacious: An Educational Intelligence Platform for Instructional Designers

Yumou Wei, Michael C Sutton, Paulo F. Carvalho

cs.CY cs.ETcs.HC ·submitted 2026-11-10 04:09 UTC · v1

Abstract

Designing effective instruction requires instructional designers to create assessment questions aligned with clearly defined knowledge components (KCs). These KCs represent specific knowledge or skills that students must acquire. This process is often time-consuming and relies heavily on expert labor. Although AI for education research has produced a growing body of automated question-generation and KC-discovery methods, few have moved beyond research prototypes into user-friendly tools that instructional designers can readily use. To bridge the research-to-practice gap, we developed and evaluated an educational intelligence platform that unifies question generation and KC discovery behind a single interface to support instructional content development. Named "Perspicacious," our platform aims to help instructional designers "see through" complex data and algorithms to derive instructional insights. A cost model calibrated on 17 production runs revealed a cost-time trade-off: extracting KCs from a 576-question bank cost US $1.98-4.11 and took 36-158 minutes to complete, depending on the budget. A formative, partially Wizard-of-Oz study with eight instructional designers found strong perceived usability (mean SUS 78.8) and surfaced concrete requirements that we treat as prerequisites for deployment.

Closest papers

Nearest neighbours by meaning across the whole index.

A Prompt-Engineering Approach to Develop Scalable, Flexible, and Real-Time Hybrid Micro-Level Personalization in a General Purpose AI Teaching Assistant

cs.AI ·2026-09-03 ·Saptarshi Basu, Sandeep Kakar, Ashok Goel
similarity 0.59
Abstract · 7 pages, 9 figures, IAAI27 conference

Artificial intelligence (AI) teaching assistants powered by large language models (LLMs) offer scalable educational support but often provide limited personalization. This study presents a prompt-engineering-based framework for personalizing general-purpose LLM/RAG-based AI teaching assistants such as Jill Watson across academic disciplines and courses. The framework adapts responses using six learner-specific dimensions: self-assessment, abstraction preference, verbosity preference, perceptual orientation, information processing style, and level of understanding, yielding 96 distinct learner profiles. Student queries are additionally analyzed using Bloom's Taxonomy to estimate cognitive complexity at the interaction level. Learner attributes and cognitive assessments are encoded in structured prompts that condition the LLM without requiring model retraining. The framework is evaluated through experiments using NLP metrics and a human study with five participants. Results show perceived differences in response style and structure across personalization conditions, with statistical analyses identifying learner attributes associated with measurable response changes. These findings provide preliminary evidence that prompt-based personalization can support adaptive behavior in LLM-powered educational agents.

0.16

StudentBench: AI and human tutoring yield equivalent GRE learning gains

cs.AI cs.CY ·2026-09-23 ·Curtis Northcutt, Inaara Hasmani, Kevin Feng, Trevor Khangi, Andreas Plesner +1 more
code/data similarity 0.59
Abstract · 47 pages, including references and appendices. Data: https://huggingface.co/datasets/handshake-ai-research/studentbench Code: https://github…

Artificial intelligence offers an unprecedented opportunity to augment human capabilities, yet progress at the frontier has focused primarily on advancing model capabilities. We introduce StudentBench, a suite of AI teaching evaluations and a public platform that enables large-scale data collection with over 175,000 student-AI messages to study whether large language models (LLMs) produce learning gains equivalent to human tutoring. Using StudentBench, we measured learning gains on Quantitative and Verbal GRE questions across 2,383 human participants receiving AI tutoring, human tutoring, or no tutoring. We establish that AI tutoring is statistically equivalent to expert human tutoring for GRE learning gains (p = .015), and in five of the seven GRE domains, the best performing AI tutor surpassed the human tutor, on average. In a second study, expert human tutors compared LLM-generated lesson plans and practice problems through 2,028 pairwise rubric evaluations. Together, the two studies clearly separate AI tutors across: (1) lesson planning, (2) practice-problem creation, (3) conversational pedagogy, (4) cost, and (5) engagement. Surprisingly, one AI tutor achieved learning gains equivalent to human tutoring (p = .044) at 918 times lower cost (USD 0.0052 for AI versus USD 4.81 for human, per percentage point gained). For Quantitative GRE sessions, faster AI replies correlated with more student messages, more messages with more correct practice, and more correct practice with larger learning gains (all p < .002). To support future research, we open-source the de-identified data collected in our studies.

0.32

Critical Thinking with Generative AI: A Constraint-First Design Pilot of a Thinking-Partner Intervention

cs.HC ·2026-09-29 ·Fatima Tuz Zahra, Jiangen He, David M. Bowers, Wei Wang
similarity 0.59
Abstract · 53 pages, 5 figures

Generative AI (GenAI) tools entered higher education classrooms faster than the field was able to study their effects on learning. One concern is that GenAI may displace the critical thinking and AI literacy that students will need after graduation. This paper reports a Design-Based Research pilot of a GenAI-assisted critical thinking framework, in which ChatGPT was used as a thinking partner in an undergraduate research methods and statistics course during Spring 2025 (N = 14). The mixed-methods design combined pre- and post-intervention measures of statistical learning (AASCDM), AI literacy (MAILS), and critical thinking (WGCTA) with instructor field notes, student artifacts, and student-AI interaction logs. Pre-post tests showed gains on every AASCDM dimension and on eight of nine MAILS dimensions, while WGCTA percentiles did not change. Qualitative analysis identified four themes: the ways students positioned the LLM (as answer generator, validator, or co-thinker); the depth of student engagement (procedural vs. conceptual); occasional humanizing of the tool; and the role of curriculum design in shaping each of the prior three. Read together, the findings indicate that one semester of GenAI-assisted instruction can move domain learning and self-reported AI literacy but does not move standardized critical thinking, and that the modal student-LLM relationship is one of validation instead of dialogue. We end with design principles for the next iteration of the framework and implications for research on adaptive and personalized learning.

0.20

Socrates went Nuclear: Comparing Interaction Strategies for AI systems in a Learning Context using Brain Sensing

cs.AI cs.HC ·2026-09-01 ·Alexandre Clin Deffarges, Nataliya Kosmyna, Pattie Maes
published similarity 0.58
Abstract · 11 pages, 2 figures, 3 tables, to appear at The 14th International Conference on Human-Agent Interaction (HAI'26)

Does unrestricted AI access bypass the cognitive effort required for learning, or does it streamline knowledge acquisition? This paper reports on a study where we compare three designs for user-AI interaction in a learning context: (1) an unrestricted conversational bot like ChatGPT, (2) a pedagogically constrained bot that guides through hints without giving final answers, which we refer to as the Socratic mode; and (3) a non-conversational adaptive tutoring system that adjusts difficulty in real-time based on the user's cognitive engagement derived from the brain signals. Fifty study participants were tasked with learning about nuclear safety protocols, a domain chosen for its zero-prior knowledge baseline. The participants progressed through an instructional video, a pre-test, an AI-driven assessment phase, which varied in the three conditions, and an immediate post-test. The nature of the questions centered primarily on factual knowledge acquisition, but it still required participants to have a global understanding of the concepts in order to answer the questions correctly. A Muse headband was used to derive the cognitive engagement of all users in all conditions. The unrestricted chatbot produced higher learning gains (delta) than both constrained modes (p < .03, d > 0.80), while the adaptive condition generated significantly higher EEG engagement (p = .018). The cluster analysis of chatbot usage and discussion patterns by users showed that most participants in the unrestricted-mode adopted a direct answer-retrieval strategy, while participants in the Socratic-mode initially attempted to reason through the hints before progressively disengaging. Consequently, this also suggests that the success of the unrestricted AI is not an evidence of deeper learning, but rather a result of the immediate post-test evaluation after the training phase.

0.58

Designing Accessible and Rigorous ML Experiments: No-Code Visual Workflows and Structured Guidance for Beginners

cs.HC ·2026-11-10 ·Henrik H. J. Detjen, David Pfrang, Max Westphal, Melissa Finster, Dörte Corr +1 more
similarity 0.58
Abstract

Designing machine learning (ML) experiments requires beginners to plan a methodologically sound workflow and implement it in a concrete environment. We evaluated two forms of support in two within-participant studies with 18 predominantly technically trained participants: mlguide, which provides structured, evidence- backed guidance for workflow design, and AI Sandbox, a visual no-code environment for workflow implementation. After using mlguide, participants produced better high-level workflow designs, but their scores on a quiz about specific methodological choices did not improve significantly. Compared with Google Colab, AI Sandbox received higher hedonic user experience ratings, while perceived workload and acceptance did not differ significantly. These findings suggest that different forms of interactive support help with differ- ent aspects of beginner ML experimentation. The paper contributes a reusable evaluation task with instructions, a modified dataset, and a quiz, as well as empirical evidence on structured guidance for workflow design and on the experience of implementing workflows in visual and notebook-based environments.

0.36

LINK: A Corpus-Grounded Design Pattern Library for Educational Technologies

cs.HC ·2026-10-02 ·Lorena Lyon, Hyoungwook Jin, Vincent Cavez, Roy Pea, Xu Wang +1 more
similarity 0.58
Abstract

Designing learning interfaces is often an ill-structured problem: effective solutions depend on learning goals, subject matter, learner characteristics, and interaction context that are difficult to generalize. Designers must integrate knowledge from learning sciences, interaction design, and subject domains when translating learning goals into interface decisions. Yet, prior efforts to connect learning theory and design have largely focused on learning activities or specific technologies, leaving interface-level design knowledge fragmented across systems. In this work, we take a bottom-up approach to surface this knowledge from existing educational interfaces. We present LINK (Linking INstructional Knowledge to interfaces), a library of 27 recurring interface patterns, organized under nine higher-level design commitments, which describe broader learning-science-grounded goals for interfaces. As part of developing LINK, we conducted a user study (N=14) with designers to evaluate LINK's generative power. Our findings suggest that LINK helps designers generate interface ideas and articulate theory-grounded rationales for design decisions.

0.18

Configuring AI for Exploratory Learning: Educator Perspectives on Inference, Persistence, and Disclosure

cs.HC ·2026-09-17 ·Si Chen, Xinyue Chen, Artur Mullagaliyev, Alexander Nwanganga, Shifu Hou +3 more
similarity 0.58
Abstract · 32 pages

Generative AI (GenAI) can support open-ended learning through generation, personalization, and learner modeling, yet educators need ways to shape these capabilities around educational goals. Through interviews and design activities with 15 U.S. educators, we examined educator configuration of GenAI using K-12 career exploration as an exploratory context. Educators configured not only AI-generated experiences, but also when student activity became an inference, whether learner information persisted, who could access it, and how it informed subsequent human action. They also faced challenges translating teaching needs into configurations: recognizing possibilities for control beyond familiar uses of GenAI, decomposing general-purpose AI into understandable functions and responsibilities, and identifying useful information through intended teaching actions. We discuss how GenAI systems can support educators in expressing and testing configurations, while establishing boundaries around personalization, inference, persistence, disclosure, and action to keep AI-supported learning aligned with evolving learner needs.

0.20

Judgment-Centred Software Engineering Education: A Post-Hype Review and Framework for AI-Augmented Learning

cs.SE cs.CY ·2026-09-24 ·Qusay H. Mahmoud
similarity 0.58
Abstract · 22 pages, 1 figure, 3 tables

Generative artificial intelligence has moved from a disruptive novelty to a recurring part of software-development and computing-education workflows, while software agents are beginning to act across repositories, command lines, browsers, tests, and other tools. The educational problem is no longer whether students should be allowed to generate code, but whether software-engineering (SE) programs can preserve and assess human understanding while preparing students to work responsibly with increasingly capable AI systems. This paper presents a structured integrative review of research and practice from 2023 through 23 September 2026, supplemented by established work on AI literacy, technical debt, and human-AI collaboration. The evidence supports a conditional conclusion: GenAI can improve access to explanations, feedback, practice, and short-term task completion, but learning outcomes depend on prior knowledge, scaffolding, verification, task design, and assessment. We examine student help-seeking and authorship, faculty assessment and policy demands, and the wider SE risks created when generated artifacts persist across repositories, teams, architectures, and deployed systems. We extend the concept of comprehension debt: the deferred learning and maintenance cost that arises when AI-assisted production outpaces a learner's or team's ability to explain, test, modify, and justify the resulting software. We then refine the AI-Augmented Software Engineering Education (AASEE) framework into five non-linear integration levels and four cross-cutting evidence obligations: explain, verify, modify, and account. The framework links delegation to evidence, governance, and recovery mechanisms appropriate to its consequences.

0.33