CS Paper Radar

An Empirical Study on how Computing Students Interact with Large Language Models for Learning Purposes

Opetunde Ibitoye, Saheed Popoola

cs.CY ·submitted 2026-09-27 16:50 UTC · v1
survey

Abstract

Large Language Models (LLM) have become popular tools in education. Therefore, understanding how students interact with LLMs for learning purposes is an important aspect of the responsible design and development of LLM based learning tools. However, existing research has focused on evaluating model performance, specific use cases, and students' attitudes toward LLM in education. There has been limited empirical work exploring students' interaction strategies with LLMs in computing education. This study draws on Self Regulated Learning (SRL) theory and conducts a survey of 225 undergraduate and graduate computing students to investigate how students use LLM tools during learning, the perceived learning impact, and the strategies they rely upon when using these tools. Students used LLMs across nine functional categories, most often to explain concepts, learn new material, find information, debug, and generate ideas. They also adopted some strategies such as supplying detailed context and iterative prompting. The most common difficulties encountered with LLM tools were incorrect information, generic responses, and difficulty trusting the model's reasoning. We synthesized these patterns through an SRL lens and offer a Probe Monitor Shape heuristic to map students' LLM interaction strategies onto the phases of self-regulated learning.

Comment
16 pages, single column, 3 figures, 2 tables

Closest papers

Nearest neighbours by meaning across the whole index.

RegulaTutor: A Theory-Informed Framework for Long-Term Personalized Learning

cs.AI ·2026-10-09 ·Yonggang Li, Boqian Han, Rui Sheng, Yushi Sun, Zixin Chen
similarity 0.68
Abstract

Long-term personalized tutoring requires language models to translate a learner's evolving history into appropriate instructional support. Existing LLM-based tutoring systems use memory and retrieval to personalize responses, but the pedagogical rationale linking historical evidence to teaching strategies is not always explicit. We introduce RegulaTutor, a theory-informed framework for long-term personalized tutoring that connects historical evidence, learner interpretation, and learner-facing instruction. Guided by self-regulated learning (SRL) research, its tutoring harness operationalizes a cognitive, affective, metacognitive, and motivational (CAMM) framework for deriving evidence-supported learner interpretations and teaching policies. The resulting policies retain links to their supporting evidence and are updated as new observations arrive. This staged workflow also supports systematic refinement through PairRoute, an offline procedure that uses intermediate execution records to distinguish harness, model, and mixed failures. PairRoute proposes and evaluates targeted harness revisions, then constructs reviewed stage-level supervision for confirmed model failures remaining under the selected harness. We focus this refinement on historical evidence acquisition, which underpins personalized instruction and allows outputs to be checked against learning records and reference answers. RegulaTutor improves historical-evidence accuracy, learner diagnosis, and judged teaching quality on average across evaluated model configurations. PairRoute results show an endpoint improvement in standalone-model historical-evidence accuracy over the refinement trajectory. Together, these results highlight RegulaTutor's potential to support lifelong personalized education through instructional policies that evolve with learner histories and targeted improvements to the tutoring system.

0.28

Who (or what) are they prompting? How CS students conceptualize and use LLMs

cs.CY ·2026-11-10 ·Nikitha Donekal Chandrashekar, Margaret Ellis, Sehrish Basir Nizamani, Naren Ramakrishnan
published similarity 0.67
Abstract · Accepted to SIGCSE 2027

Large language models (LLMs) are increasingly used by students in computing courses. Yet computing education research has not fully examined how teaching and inclusion of LLMs shape students' conceptualization and understanding of LLMs, and how these factors relate to their subsequent use. Although prior work has examined prompting strategies and AI literacy, the relationships among students' conceptualizations of LLMs, their understanding of LLM mechanisms, and their prompting behavior remain underexplored. We studied these relationships in a post CS1 course that introduces students to the various CS subfields. We adapted the course to teach and allow the usage of LLMs. We administered an end-of-course survey, using open-ended items to elicit students' representations of LLMs and their explanations of how LLMs work, alongside closed-response measures of prompting style and prompt count (N=283 students consented and provided codable responses). Using qualitative coding, we categorized students' understanding of how LLMs work into four ordinal knowledge levels based on the Structure of Observed Learning Outcomes (SOLO) taxonomy and coded their metaphoric conceptualizations along two dimensions: persona type and relational role. Our results show that students most often conceptualized LLMs as human assistants, and context-rich prompting was the preferred style. The study contributes a two-dimensional characterization of students' LLM metaphoric representation and exploratory analyses also identified associations among these constructs showing how understanding these systems relates to their usage.

0.43

Understanding Student Use of Large Language Models Across Computer Science Subfields

cs.CY cs.HC ·2026-10-01 ·Sehrish Basir Nizamani, Yoonje Lee, Nikitha Donekal Chandrashekar, Margaret Ellis, Naren Ramakrishnan
published similarity 0.64
Abstract · Accepted to the 2026 IEEE Frontiers in Education Conference (FIE 2026). 3 figures, 4 tables. (c) 2026 IEEE. Personal use of this material is…

This research full paper examines how undergraduate students use large language models (LLMs) across computer science subfields. As LLMs become increasingly integrated into computing education, understanding how their use varies across technical and pedagogical contexts is essential for designing effective, subfield-aware instruction. This paper presents a cross-subfield analysis of LLM usage among 211 undergraduate students in a problem-solving course intentionally designed to support responsible and effective LLM use through structured instruction and reflection. Using post-assignment reflection data collected across seven instructional modules spanning multiple computer science subfields, we examine prompt counts, LLM role conceptualization, and verification behavior. Results show that LLM adoption varies substantially by assignment, with higher usage in algorithms and web development and lower usage in software engineering. Students predominantly treat LLMs as assistive tools rather than authoritative sources, and verification is common across all subfields, with most students using multiple strategies. Verification behavior also varies by assignment context, with testing more common in structured tasks and web search more common in open-ended tasks. These findings suggest that assignment characteristics play a central role in shaping how students interact with and evaluate LLM outputs, even under a single, consistently applied instructional design. This work contributes empirical evidence on how LLM adoption, role conceptualization, and verification behavior vary across computer science subfields, extending our prior work on structured, reflective LLM instruction to show how its effects differ by task rather than only in aggregate.

0.30

Student-LLM Interactions During Math Learning: Limited Epistemic Proactivity Overall, but Within-chat Change Predicts Performance Without AI

cs.CY ·2026-10-28 ·Rania Abdelghani, Peter Kaiser, Kou Murayama
dataset similarity 0.62
Abstract

Large language models (LLMs) are increasingly used for learning, raising a key question: can learners remain epistemically proactive, retaining responsibility for the cognitive and regulatory activities through which learning is produced while working with AI? Because this responsibility may change as interactions unfold, we examine epistemic proactivity as a developing process. We conceptualize epistemic proactivity through self-regulated learning (SRL) moves, complemented by help-seeking strategies and manifestations of epistemic vigilance. We analyzed these behaviors in 880 substantive turns from 112 secondary-school students completing open-ended mathematical-modeling practice with a general-purpose LLM (Mistral). Overall, students showed limited observable epistemic proactivity: their interactions were strongly request-dominated, with planning, monitoring, and evaluation relatively rare, although low-quality AI responses elicited more explicit evaluation. Temporal analyses qualified this picture: explicit SRL-related moves increased modestly over the chat, but help-seeking strategies became less understanding-oriented. Crucially, these temporal trajectories in help-seeking strategies predicted their post-test performance without AI beyond pre-test and aggregate patterns: students whose requests became more understanding-oriented performed better once AI assistance was removed. Thus, while students had a limited epistemic proactivity overall, its temporal development still was a critical predictor of post-test performance without AI. These results underscore the importance of examining the temporal organization of epistemic proactivity, rather than isolated prompt types, for understanding AI-supported learning. We release our fully-anonymized dataset publicly in an OSF repository.

0.11

What the submission does not show: Complexifying cognitive offloading in students' mathematical work with LLMs

cs.CY math.HO ·2026-10-22 ·Igor' Kontorovich, Miriam N. Wallach
similarity 0.62
Abstract · 44 pages. Preprint under consideration at Digital Experiences in Mathematics Education; not peer reviewed

A common concern is that using large language models (LLMs) lets students offload mathematical work that tasks are designed to elicit. Yet such claims often rest on outcomes (e.g., submitted solutions), rather than on evidence of students' work with LLMs. We followed 23 first-year linear algebra students as they solved three differently designed homework tasks with LLMs. Drawing on self-regulated learning and choice-affluent environments, we interpreted each of 429 prompts as a trace of a regulatory event and compared 59 chatlogs with the problem solutions the students submitted. Prompts formed five structures, casting the LLM mainly as a solver, explainer, or validator. The mix of structures shifted across the tasks, and ten of the 15 students who completed all three tasks changed their dominant structure. Most final submissions stayed close to LLM output, including those that followed exchanges in which students had explained, conjectured, and checked their own ideas. We conclude that an LLM-aided submission is not a trustworthy marker of students' offloading of the mathematical work, and that offloading needs to be examined in each regulatory space and part by part rather than as a whole.

0.28

Cognitive Mapping of Learning by Thinking Mechanisms in Large Language Models

cs.AI ·2026-11-10 ·Dong Han, Jin Zhang, Kunyue Yang
similarity 0.62
Abstract · Theoretical framework paper; 24 pages, 3 tables, 1 figure

Large language models (LLMs) increasingly exhibit capabilities that resemble human Learning by Thinking (LbT), the acquisition of new insight through internal processes such as explanation and reasoning, without new external input. A systematic account of the cognitive mechanisms behind these capabilities, and of their limits, remains underdeveloped. This paper proposes a Cognitive LbT Framework that maps four mechanisms (Reflective Reasoning, Synthetic Memory Formation, Internal Simulation, and Conceptual Self-Organization) onto computational processes in LLMs. Drawing on representative methods, including chain-of-thought prompting, self-consistency, self-reflection, Tree-of-Thoughts search, Self-Instruct, and self-taught reasoning, the analysis argues that selected methods can functionally approximate aspects of these mechanisms and can improve performance on some tasks without additional labeled examples, although this does not always constitute durable learning in model parameters. For example, Self-Instruct generated approximately 52,000 instructions from 175 seed tasks and reported a 33% absolute improvement on Super-NaturalInstructions (Y. Wang et al., 2023). The same mechanisms also produce characteristic failures, including hallucination diffusion, model collapse, concept drift, and high computational cost. These failures are not isolated technical defects but follow from four structural asymmetries between LLMs and human cognition: the identity of generator and monitor, closed learning loops without external grounding, symbolic rather than sensorimotor simulation, and the cost of inference-time computation. The framework yields testable propositions about when LbT in LLMs should succeed or fail and offers a structured basis for designing more reliable autonomous learning systems.

0.16

Penquiry: A Pen-based Interactive In-situ Q&A System Leveraging LLMs

cs.HC ·2026-09-17 ·Jeongmin Rhee, Changhee Lee, Hyunwoo Kim, Kiroong Choe, Bohyoung Kim +2 more
top venue similarity 0.61
Abstract · 19 pages, 12 figures, Accepted to ACM UIST 2026

Pen-based digital devices remain a preferred medium for active, cognitively engaging study. Concurrently, Large Language Models (LLMs) have become indispensable for self-directed learning, enabling students to clarify concepts. However, a fundamental interaction gap exists between the fluid, spatial nature of pen-based workflows and the discrete, keyboard-heavy requirements of LLMs. We present Penquiry, an in-situ question-and-answer system that bridges this gap by enabling learners to pose questions directly on digital study materials via a pen. We characterize two primary interaction challenges in this multimodal transition: a Referential Barrier, which hinders grounding fine-grained visual elements into the query context, and an Expressive Barrier, which forces learners to translate diverse, non-textual intents--such as equations and diagrams--into rigid, typed sentences. To resolve these, Penquiry introduces a mediation layer featuring Content Snapping for unambiguous referencing and Question Autocompletion to expand sparse ink keywords into rich semantic queries. Through two iterative user studies (N = 16 per study), we demonstrate that Penquiry significantly reduces the cognitive and physical overhead of inquiry compared to traditional interfaces, providing a new blueprint for pen-based, in-situ AI interaction

0.33

Self-Explanation Tutor for Active Study of CS1 Worked Examples

cs.CY cs.AIcs.HC ·2026-08-25 ·Arun-Balajiee Lekshmi-Narayanan, Mohammad Hassany, Kamil Akhuseyinoglu, Rully Hendrawan, Peter Brusilovsky
similarity 0.60
Abstract

Worked examples are an important part of introductory programming, but reading their expert explanations is passive. Self explanation, students explaining the problem and its solution to themselves with subgoal level analysis, converts passive reading into an active study of worked example, yet it is hard to scale because assessing free-text explanations and returning timely feedback has had no easy automated solution. We investigate whether a large language model (LLM) can fill that gap. We build a self-explanation tutor for introductory programming, ESSE, in which students explain lines of worked examples and receive immediate LLM feedback on the correctness and completeness of each explanation, and we pursue two goals. First, we ask whether the LLM judges student explanations well enough to serve as the engine of the tutor; we assess its judgments against two independent human reference standards of different kinds, a single domain expert and a crowd of non-expert raters, each with its own strengths and weaknesses, characterizing both where the LLM is reliable and the systematic tendencies in how it diverges. Second, we ask whether the LLM-based tutoring benefits students; deploying it in an introductory Java course, we find that its feedback leads students to persist and revise rather than abandon a line, that their explanations grow more complete and conceptually richer across attempts, and that students show evidence of learning. These indicate that LLM-based assessment is good enough to power a self-explanation tutor, and that the tutor positively shapes how students study worked examples.

0.10