CS Paper Radar

Reducing Barriers to Academic Support: Evaluating a Course-Specific RAG System for Addressing Help-Seeking Disparities in Higher Education

Andy Gray, Jake Hobbs

cs.AI cs.CYcs.HC ·submitted 2026-09-18 10:30 UTC · v1

Abstract

Access to academic support is a key determinant of student success, yet students experience it unequally: some readily seek help from lecturers or tutors, while others hesitate due to anxiety, fear of judgement, uncertainty about expectations, or low confidence in their understanding. This may be especially evident in computing education, where programming tasks are cumulative and cognitively demanding. Although students increasingly turn to general-purpose generative AI tools, these can produce responses that are inaccurate, insufficiently contextualised, or misaligned with module expectations. This study presents and evaluates Beacon, a course-specific Retrieval-Augmented Generation (RAG) system providing private, immediate, module-aligned academic support. Grounding responses in approved teaching materials, Beacon was designed to lower barriers to help-seeking while encouraging independent learning. Using a design-based research approach, Beacon was developed iteratively and evaluated via mixed methods, combining questionnaires and semi-structured interviews with students and staff at a Higher Education institution. Students described Beacon's responses as closely aligned with module content and more trustworthy than unrestricted generative AI tools, valuing its use of pseudocode and scaffolded explanations over direct solutions. Although participants remained cautious about trusting AI-generated responses without verification, they viewed the system as a valuable first point of support before consulting lecturers or official resources. The findings suggest that carefully designed course-specific AI systems may reduce barriers to academic support by occupying an intermediary space between independent study and formal support. Rather than replacing educators, educational AI may be most valuable when it broadens access to guidance while preserving the pedagogical role of lecturers.

Closest papers

Nearest neighbours by meaning across the whole index.

HelpCoach: Scaffolding Targeted AI Help-Seeking During Problem-Solving

cs.HC ·2026-09-24 ·Hyoungwook Jin, Weirui Peng, Jieun Han, Q. Vera Liao, Xu Wang
similarity 0.66
Abstract

Students increasingly turn to AI for help with problem-solving, yet too much AI support can undermine learning itself. To benefit from AI, students need to specify the necessary knowledge and scaffold type in their questions. However, they struggle to formulate such targeted questions because they lack metacognitive skills to recognize and select effective help options. We developed HelpCoach, an add-on for chat interfaces that helps students formulate knowledge- and scaffold-specific questions and receive targeted help during problem solving. HelpCoach continuously assesses students' help-seeking performance and prompts students to improve through an adaptive revision template. Whereas prior work has largely taught help-seeking skills apart from learning tasks, HelpCoach's in situ scaffold enables concrete practice on metacognitive skills and immediate revisions to help-seeking behavior. In a study with 40 college students learning web programming, HelpCoach led to more specific questions during chatbot interactions and greater knowledge retention than pre-task help-seeking training alone.

0.17

The Uneven Impact of Generative AI on Student Learning: Examining the Roles of Reliance, Evaluation Literacy, and Course Policy in AI-related Courses

cs.AI cs.ETcs.LG ·2026-09-16 ·Lydia Manikonda, Mei Si, Sirajam Munira, Oshani Seneviratne, Kristin Bennett
hot direction similarity 0.66
Abstract · Paper under review

Generative artificial intelligence (GenAI) is changing how students learn, yet the roles of course context, cognitive reliance, evaluation literacy, and early reliance remain underexplored. Using survey responses from 118 students across 12 AI-related courses at our institution, we examined differences in GenAI use and perceived learning experiences. We identified four user clusters: high-use students reporting many benefits, light users reporting less reliance and fewer benefits, and two moderate-use groups reporting different levels of benefit. We also found significant differences between free- and premium-version users, single- and multiple-tool users, and students experiencing different instructor policies. In multivariable regression models, academic benefit was associated with early reliance and academic task support; positive impact was associated with cognitive reliance, academic task support, confidence in GenAI reliability, and instructor policy; and negative impact was associated with early reliance and attitudinal change. The association between early reliance and negative impact became stronger as evaluation literacy increased. Finally, perceptions of GenAI-enhanced learning appear to reflect cognitive, performance, and self-efficacy benefits, while concerns about stress and diminished critical thinking are associated with lower perceived learning benefits. These findings suggest that institutions need better policies to address such inequities so that institutions can enable students to benefit from increasingly capable AI systems.

0.35

LLM Pedagogical Behavior in AI Tutoring Interactions

cs.CL cs.HC ·2026-08-24 ·Suhyeon Lee, Juneha Baek, Jaehyeong Park, Donghyuk Shin
top venue similarity 0.65
Abstract

Students increasingly use LLMs as tutors for coursework and problem solving. Little is known about the level of assistance LLMs provide when students use them as tutors in authentic learning interactions. This matters because tutoring responses can differ substantially in how directly they help students complete a task. We operationalize this dimension as scaffolding level and develop a five-level scale, validated against human annotations, that characterizes responses according to the degree of direct assistance they provide. We apply the scale to 14,637 LLM responses from 203 students in a university AI course. Responses are overwhelmingly concentrated at high levels of assistance, with more than 95% classified as either Explaining or Solving. Scaffolding level is systematically associated with students' subsequent conversational behavior, but provides little additional predictive information about performance on three subsequent exams beyond prior achievement and dialogue behavior. These findings provide an empirical baseline for LLM assistance in tutoring interactions and a measurement framework for evaluating how alternative tutoring designs change that assistance.

0.46

On-Premises Multi-Course RAG Tutoring for Business Education: Hardware-Software Trade-offs in a Campus AI Tutor

cs.AI cs.IR ·2026-10-01 ·Sidney Shapiro, Joshua Lindemann
similarity 0.64
Abstract · 23 pages, 3 figures, 4 tables

Campus AI tutors based on retrieval-augmented generation (RAG) must ground answers in assigned course materials while keeping textbooks and student dialogue on institutional infrastructure. We present CourseChat, an on-premises, multi-course RAG tutor for undergraduate business education, deployed behind a campus web gateway and intended for use embedded in Moodle. Six isolated course offerings, each keyed by its own course reference number (CRN), share twin-edge AI hosts running a FastAPI service, a local vector database, and a local large language model (LLM) served by Ollama. We report two generation-model bake-off rounds, a separate fixed-evidence source-fidelity comparison, and conversation and quiz audits. Several larger models failed the classroom speed gate, but a 12B model and a 7B alternative passed. A separate mixture-of-experts candidate improved some corrections while introducing new factual and continuity errors. We therefore retain the 8B production model pending a demonstrated overall improvement, rather than claiming that 8B is universally optimal. Software changes improved follow-up topic resolution while preserving course scope; 435 prebuilt questions across 65 modules decouple practice from live generation. The results support treating model choice, evidence selection, serving compatibility, and product design as a joint engineering decision. They do not establish learning gains: faculty ratings, peak-load capacity, and complete public-gateway acceptance remain separate evaluation needs.

0.16

LearnAI: Just-in-Time AI Co-Creation Across Disciplines at a University

cs.CY cs.HC ·2026-08-19 ·Weihao Qu, Ling Zheng, Chris Buzaid, Daniel Crawford
publishedhot direction similarity 0.64
Abstract

As generative AI reshapes professional and educational practice, institutions face a challenge: how to support diverse learners, from non-coders to advanced students, in building confidence and practice with AI-supported problem solving. Most institutional responses bifurcate into conceptual workshops for general audiences or technical courses for computer science majors, leaving few spaces where mixed-ability learners can engage common AI tasks at levels matched to their prior experience. This experience report presents the LearnAI Framework, a two-layer model for just-in-time AI co-creation piloted at a comprehensive teaching university. The Wide-Exposure Layer embeds short presentations in existing courses to build AI awareness at scale, reaching students and faculty across 18 courses in five disciplines. The Customized Co-Creation Layer provides opt-in, one-on-one sessions where clients work with trained undergraduate tutors through a 5-Stage Pedagogical Script: Problem Framing, Tool-Task Mapping, Iterative Co-Prompting, Deployment and Verification, and Ethical Reflection. Over two semesters, 35 clients co-created 36 portfolio websites and over 20 deployed web applications. Interviews with five clients and two tutors suggest a recurring change in how clients described AI use, shifting from treating AI as a passive answer machine to engaging it as a collaborative tool under human direction. A small paired pre/post AI readiness dataset (N = 7) provides preliminary descriptive context, and tutor accounts document how the pedagogical script was enacted and adapted across client types. We report on boundary cases including clients who felt overwhelmed and respondents who deliberately rejected AI use. This paper contributes a practical, adoptable framework with initial evidence from a single institution.

0.38

Examining Variation in How Guided AI Tutors Resolve Student Impasses

cs.AI ·2026-09-29 ·Bakhtawar Ahtisham, Kirk Vanacore, Alessandra Napoli, Josh Arens, Ksenia Ionova +3 more
similarity 0.63
Abstract

When a student is stuck, a tutor faces the assistance dilemma: help given too early can hinder productive struggle, while help withheld too long leaves the student in a frustrating, persistent impasse (i.e., wheel spinning). Generative AI tutors increasingly use guardrails restricting answer-giving, yet little is known about how such tutors behave once an impasse persists. We analyze 20,462 student turns from 1,260 authentic sessions with a guided LLM chemistry tutor, identifying 6,630 impasse turns of three major types: conceptual errors, expressed uncertainty, or help-seeking. We then used these impasses to simulate three tutoring conditions to study variation in AI tutor guidance through impasses: baseline, no-direct-answer, and guided tutor. For a sample of 150 impasses, prompt specificity changed pedagogy: a baseline tutor provided the answer directly in 50.7% of responses, a no-direct-answer tutor asked a follow-up question every time, and the guided tutor responded in a wide variety of ways depending on the context. We then analyzed impasse trajectories in authentic interactions, finding that each additional impasse turn lowered the odds of next-turn recovery by 12.7% (AOR = 0.873, p < .001), and early dropouts were caught in recursive concept elicitation before reaching execution. The benefit of questioning decayed as impasses persisted (scripted question x depth AOR = 0.78; follow-up x depth AOR = 0.83), whereas addressing the student's error grew more beneficial (AOR = 1.14); after a failed scripted question, repeating it was followed by recovery in 28.1% of cases, compared with 39.8% when the tutor addressed the error instead. For learning analytics, these findings identify impasse depth and type as observable, turn-level dialogue signals that analytics can use to trigger graduated, state-sensitive assistance in real time.

0.13

AI literacy over tool design: a mixed-methods study of scaffolded versus unrestricted generative AI in programming education

cs.CY ·2026-09-15 ·Sepinoud Azimi
similarity 0.62
Abstract

Generative AI has become a routine resource in programming education, and most institutional responses to it are attempts at control, either by restricting access or by offering students a controlled version of the technology. This paper reports a seven-week mixed-methods pilot study in a master's-level data analytics course, in which 33 students were randomly assigned either to a scaffolded AI Study Coach embedded in the notebook-based laboratory sessions or to unrestricted use of AI tools of their own choosing. The Coach offered stepwise hints, did not generate code, limited the number of hints per session, and required a short reflection at the end of each session. The design assumed, in line with scaffolding theory and recent experimental evidence, that guided and limited support would build confidence and reduce over-reliance, and that the scaffolded group would learn more. Assignment performance did not differ between the conditions. Students in the Coach condition reported higher confidence but managed the hint budget poorly, while students in the unrestricted condition were satisfied with their tools and uneasy about how much they depended on them. In interviews, students in both conditions identified awareness of their own reliance on AI as the most valuable outcome of the course. Students who had formulated their own rules for when to use AI performed better in both conditions, and those with the best understanding of how the models work, in every case self-taught, used the tools most deliberately and achieved the highest scores. The design of the tool mattered less than the students' capacity to govern their own use of it, a capacity that is at present acquired by chance. The paper argues that the appropriate response is structural: assessment that grades the reasoning behind AI-assisted work, and AI literacy taught explicitly as a core skill.

0.27

Edhivo: An AI-Native Platform for Scalable Assessment and Instant Feedback in Higher Education

cs.CY ·2026-10-27 ·Moraish Kapoor, Jiawei Zhou
similarity 0.62
Abstract

Scaling assessment in higher education faces two intertwined bottlenecks: the cognitive burden of authoring assessments and the inability to semantically evaluate open-ended responses, which together create a persistent feedback gap that hinders student learning outcomes. To address these challenges, we present Edhivo, an AI-native pedagogical platform that addresses both bottlenecks through a vertically integrated system for content delivery, assessment, and reporting. Edhivo employs a Dual-LLM pipeline: one model generates context-aware assessments for both multiple-choice and open-ended questions, from instructor-provided materials, while a second model performs multi-criteria, rubric-based evaluation of student responses, enabling near-instant qualitative feedback at multiple hierarchical levels. All AI interactions are source-grounded to reduce hallucination risk, and a Human-in-the-Loop review process preserves instructor authority. We demonstrate Edhivo's efficacy through a pilot deployment at Stony Brook University in a real classroom environment in 2026 spring, where it supports high-frequency assessment creation and evaluation for a cohort of 36 students throughout a full semester, significantly reducing authoring effort and feedback latency while enabling novel pedagogical designs centered on active learning.

0.17