CS Paper Radar

Insights on Student Learning from Live Classroom Polls: More Than Right or Wrong

Rohit Sharma, Pavani Ayinampudi, Aditya B. M. V., Jinal Gupta, Prakash Hegade, Sakshi Sharma, Meenakshi V, SRS Iyengar

cs.HC cs.CY ·submitted 2026-09-30 04:41 UTC · v1

Abstract

A live classroom poll is usually read through its answer key, as a count of participation and correctness. Yet the same record holds more before any key is fixed, namely who answers early and how the class divides across the options. Whether these key-free signals are stable, and what they reveal, has not been examined at the scale of a real course. We study them in a live deployment of 340,668 submissions across 603 polls in 47 sessions from roughly 2,400 learners, reading each poll for a learner's response order and for how much the class divides. A single further step brings in the author-designated key, on items with a settled answer, to interpret the divided ones. A learner's response order is a stable individual signature. It reproduces at a corrected split-half reliability of 0.92, holds at 0.87 within single sessions, and is unrelated to whether the learner is correct. Read without a key, about one poll in five does not reach a clear majority, and division marks the questions that split the class. On the settled-answer items the majority is itself wrong on 49 of 358. The key-free measure flags most of these, though division alone does not separate a collectively wrong answer from a merely hard one. The two readings are largely independent, meeting at one point where early responders anticipate the eventual majority only on questions the class has nearly settled. Together they map the learners and questions of a session without a key.

Comment
8 pages, 6 figures, Submitted to ICTIEE 2027 and currently under peer review

Closest papers

Nearest neighbours by meaning across the whole index.

An Audit of Measurement Quality and Answer Bias in a Large Classroom-Poll Corpus

cs.HC cs.CY ·2026-09-30 ·Rohit Sharma, Pavani Ayinampudi, Aditya B. M. V., Jinal Gupta, Prakash Hegade +3 more
hot direction similarity 0.77
Abstract · 14 pages, 6 figures, 2 tables

Real-time classroom polls are widely used and increasingly generated with automated assistance, yet the questions themselves are rarely evaluated as measurements. We audit a large corpus of authentic classroom polls, 604 items across 47 sessions answered 340,668 times by 2,807 learners, as a measurement instrument. For the 539 items whose correct answer could be established and verified from the lecture transcript, we place every item and every student on a common scale using item response theory and analyse the answer structure of the True/False items. Two findings emerge. First, the polls form a coherent but easy scale of moderate precision (marginal reliability about 0.60), on which roughly a quarter of items barely separate stronger from weaker students. Second, students show a robust tendency to answer True, present at the individual level (77% of students lean True), which meets a milder tendency for items to be keyed False; as a result answer direction predicts difficulty, False-keyed items being about thirteen points harder, and the effect survives controls for item content and for selective answering. Both findings rest on signals a polling system already records, so the same checks can be run as items are generated, before they reach students.

0.38

Characterizing Questioning Patterns and Student Engagement Through Contextual Analysis of Real-Time Classroom Interactions

cs.HC cs.AIcs.CY ·2026-09-30 ·Rohit Sharma, Pavani Ayinampudi, Aditya B. M. V., Jinal Gupta, Prakash Hegade +3 more
publishedsurvey similarity 0.69
Abstract · 16 pages, 5 figures, One version is accepted at T4E 2026

Real-time classroom polling is now routine, yet the data it produces is usually read narrowly, as a correctness score or a headcount. Such readings say little about what a poll is doing within a lecture or how it shapes engagement. This is particularly relevant for short-response formats such as True/False, where the same question format can be used to test recall, check comprehension, or direct students' attention to a deliberately misleading statement. This study asks whether a poll's answer and instructional function can be determined by reading it against its lecture transcript, what cognitive levels of Bloom's taxonomy and instructional-function clusters the corpus contains, and how student engagement relates to answering correctly. We analyse a naturalistic corpus of 47 live sessions over 39 days, comprising 604 poll questions and 340,668 responses from 2,807 learners, most items True/False, read against time-aligned lecture transcripts and attendance. Reading each poll in context proves essential: the answer to 89% of polls is locatable in the lecture, and a recurring attention-checking device is visible only through context. Questioning is overwhelmingly lower-order and falls into seven instructional functions, and a poll's response follows its function rather than its wording. Engagement is broad but concentrated, and the class majority answers correctly 88.5% of the time, though a small set of high-consensus yet incorrect answers cannot be detected by agreement alone. An independent survey of 579 students agrees on what the polls are and on their participation, but reveals a gap between perception and reality: students cannot judge their own correctness, and the polls they find hardest are not those they answer worst.

0.41

From Source-Based Quizzes to Reteaching Decisions: A Practitioner Guide to ClassQuest

cs.CY ·2026-11-02 ·Zihao Sun
similarity 0.62
Abstract · 8 pages. Author preprint; submitted to ACE 2027, decision pending

A source-based quiz can help a computing teacher identify material to explain again, but its report must preserve the limits of the evidence behind that decision. We present ClassQuest, a prototype connecting teacher-reviewed question drafts, private student responses, topic reports and targeted retries. A Year 10 classroom account of an earlier prototype supplies 20 usefulness ratings from 24 invitees, with a mean of 8.0 and median of 9 on a ten-point scale. A Boolean-logic episode documents how immediate feedback, a teacher explanation and a fresh question were combined. We separate this account from a completed technical audit and reusable teaching guide. An exhaustive single-item audit found no count mismatches across 5,456 constructed response configurations, while diagnostic cases exposed a retry increase caused entirely by changing participation and a progression suggestion based on one learner's three answers. A further case shows that an unoffered retry can be labelled missing. A separate reporting companion distinguishes opportunity to respond, pending review and matched retry cohorts. Its completion-sensitivity check matched an independent oracle in 46,375 constructed states across roster sizes 1-30, identifying when unknown responses can change a reporting band. Our contributions are a source-linked formative workflow, executable examples of interpretation failures, and reporting and teaching materials for examining those failures. Neither the questionnaire nor the audit establishes improved learning; the findings identify reporting requirements and practical directions for formative teaching.

0.13

Who Is Sycophantic to Whom? Confident LLMs, Human Deference, and the Collapse of Trust Calibration

cs.AI cs.HC ·2026-10-22 ·Deepthika S, Pawni Yadav, B. Sankar
similarity 0.62
Abstract · 49 pages; 32 figures; 7 tables

Research on sycophancy asks whether large language models (LLMs) bend towards what their users want to hear. We ask the reverse question: when an LLM is confident, consistent, and wrong, do users bend towards it? In a within-subjects experiment, 32 participants learned up to five science topics by chatting with five assistants built on the same model (GPT-5-mini). Unknown to participants, each assistant's briefing had 0, 25, 50, 75, or 100% of its facts replaced by plausible falsehoods, and every assistant spoke with full confidence. A surprise quiz after each topic offered the correct fact, the planted falsehood, and two distractors, with a confidence rating for every answer. Across 104 quizzes and 1,040 answers, scores fell from 7.4 to 2.9 out of 10, and the planted option was chosen on 60% of falsified items against 11% of correctly taught ones (odds ratio 12.5). Transcript coding showed that whenever the assistant stated a falsehood, it was adopted about 65% of the time at every falsification level. Absorbed falsehoods were held with high confidence (8.4/10), the modeled confidence advantage of right over wrong answers vanished, and the discrimination of confidence for correctness fell from an AUROC of 0.70 to chance, while post-task trust ratings barely moved. Participants tended to return to the chat more when the assistant was worse, but returning went with more absorption, not less, and 30% of all turns pasted quiz questions back to the source being checked. The assistant itself was almost never sycophantic and never conceded under pushback. We introduce a baseline-corrected Human Deference Index to quantify this human-side sycophancy and argue that trustworthiness is a property of the human-AI pair, that sycophancy evaluations should measure both directions, and that learning interfaces must make reliability legible claim by claim.

0.26

Blind Spots Travel, Knowledge Stays Home: What RL Inherits from an LLM Judge

cs.CL cs.AIcs.LG ·2026-11-18 ·Liang Yue, Eric A. Vance
code/data similarity 0.59
Abstract · 31 pages, 6 figures, 27 tables. Code and data: https://github.com/liyu2195/blind-spots-travel

LLM judges serve as rewards where no answer key exists, in the hope that their knowledge passes to the trained model. In 814 RL runs that teach 7-8B models to answer closed-book questions or abstain, we ask what passes. What a grader knows about single answers stays home. A wrong answer that a judge rewards is memorised for that one question. A model trained with the answer key, or with a far stronger judge, chooses which new questions to answer no better than the untrained model does by sampling 32 answers and abstaining where they disagree; the grader's knowledge adds about one corrected answer per hundred questions. What a grader believes about a kind of question travels. A judge that rejects eleven of the twelve correct training answers about capitals of former states teaches the model to refuse 98% of such capitals on new questions, a loss that accuracy and abstention rates do not reveal. Changing only those twelve verdicts by hand moves the refusal from 5% to 90%, even for a grader that accepts as many answers as the answer key; at that leniency, two of four judges already cost 38-42% of these capitals. The capitals are not forgotten: with "I don't know" blocked, the model answers 94% of them correctly. The grader therefore needs no knowledge of its own: rewarding the model for giving the answer that most of its own untrained samples agree on, with no labels and no judge, earns the answer key's reward and keeps the category.

0.44

When Attention Guardrails Become Barriers to Learning: Towards the Tipping Point

cs.HC cs.CY ·2026-09-30 ·Meenakshi V., Pavani Ayinampudi, Aditya B. M. V., Jinal Gupta, Prakash Hegade +3 more
position similarity 0.59
Abstract · 10 pages, 2 figures, 4 tables

Online learning offers flexibility but lacks the structure of a classroom, where a teacher's presence guides attention. The platform we study restores that structure by monitoring the learner through the webcam during ordinary coursework, interrupting or restarting a video when the learner appears distracted. What such monitoring does to a learner across a whole course, rather than in a single examination, is largely unexamined. We report a convergent mixed-methods study of one monitored course pipeline in a summer internship program. We read two free-text surveys alongside three platform channels. The dropout exit survey yielded 15 analyzable responses; the persisting-learner reflection survey, 36. The channels are camera-verification telemetry (14,529 flags from 448 students), an in-video emotion widget (615 submissions from 273 students), and a mandatory end-of-course survey (up to 634 respondents per item). Neither survey named monitoring, so every mention analyzed here was raised by the respondent. Focus-monitoring was raised by 18 of the 51 free-text respondents: 7 of 15 dropouts and 11 of 36 persisting learners. Among the dropouts who raised it, focus-monitoring was the stated primary cause of departure in 4 of 7 cases. None of the 18 questioned being observed in principle. What learners contest is the misreading of ordinary actions, drinking water or moving the head, and the severity of what follows a flag: a video already watched returns to the start of its segment. These findings identify two targets for redesign: the severity of the response to a flag, and the environment check, which can flag a learner before any content has been seen. We argue that the proportionality of that response marks the point at which an attention guardrail becomes a barrier to learning, the tipping point this study approaches.

0.33

"I Wrote, I Paused, I Rewrote": A Mixed-Methods Study of Learner Perceptions of Process-Aware LLM Feedback

cs.HC cs.CL ·2026-09-04 ·Samra Zafar, Shifa Yousaf
similarity 0.57
Abstract · 8 pages, 1 figure, 6 tables

Large language models increasingly provide formative writing feedback, but most systems evaluate only the submitted text. Process-aware systems can additionally incorporate intermediate drafts and interaction records, yet access to these traces does not establish whether learners perceive the resulting feedback as useful, fair, or aligned with their own revision experience. We conducted a convergent mixed-method post-task evaluation with 43 undergraduate English-language learners. Participants completed a timed essay, received LLM-generated feedback based on the final essay, intermediate snapshots, and keystroke-derived records, and then answered six five-point Likert items and four open-ended questions. Ratings were concentrated above the neutral midpoint across all six items. Between 74.4% and 95.3% of participants selected ratings of 4 or 5, and all six one-sided exact sign tests remained significant after Holm correction. Perceived correspondence between the revision analysis and the participant's reported thought process was positively associated with overall perceived usefulness, Spearman's $ρ_s=.327$, 95% bootstrap CI $[.009,.609]$, $p=.032$, $n=43$. Qualitative responses were analysed through an LLM-assisted codebook development procedure followed by independent human coding of 14 binary codes. Participants frequently described the product feedback and revision rationale as accurate, while a smaller set reported misinterpretation of surface corrections, time constraints, or rhetorical intention. The findings distinguish perceived detection of revision activity from perceived interpretation of why revision occurred.

0.26

MisconceptDialogue: Evaluation Criteria Shape Measured Success in LLM Math Tutoring

cs.CL ·2026-10-01 ·Woojin Lee, Yoonkyu Woo, Jinah Kim, Hyeok-Min Gwon, Jin-Xia Huang
publishedbenchmark similarity 0.54
Abstract · Accepted to the AACL-IJCNLP 2026 Main Conference. 20 pages, 1 figure, 16 tables

Large language models are increasingly used as math tutors, raising two questions: can a single LLM detect a student's error, diagnose the underlying misconception, and remediate it through multi-turn dialogue - and can we trust the numbers that report how well it does? We address both with MISCONCEPTDIALOGUE, a fully automated framework that runs the full pipeline on five open-weight LLM tutors (7B-14B) interacting with LLM-simulated students over 5,000+ dialogues - a deliberately simulated protocol (paired conditions, a wrong-diagnosis control unsafe with real learners, scale) whose results do not establish real-student learning. On measurement, our headline finding is uncomfortable for the field: benchmark-style exact-match scoring common in standard math NLP - a lower-bound stress test - reports 42.1% measured Remediation Success Rate on identical dialogues where a human-calibrated LLM-as-judge reports 81.3%, a 39.2 percentage-point spread larger than the combined effects of tutor model, teaching strategy, and supplied diagnostic information. This is not a single-judge artifact: four independent LLM judges and 200 human annotations converge to within a few points (kappa up to 0.91). On capability, the verdict is equally striking: the score barely changes whether we tell the tutor the student's actual misconception or a deliberately swapped one - under the tested diagnosis-presentation format, these tutors show little evidence of reliably conditioning on the diagnostic information they are handed. For LLM tutoring evaluation to mature, multi-criterion scoring with multi-judge convergence and human anchoring should become standard practice - and the measurement question deserves as much scrutiny as the capability question it claims to answer.

0.32