CS Paper Radar

Critsly and StudioCrit: An Artefact-Aware AI Critique Workspace and Simulation-Based Readiness Study for Design Education

Nizam Kadir

cs.HC ·submitted 2026-09-07 01:12 UTC · v2

Abstract

Critique in design education depends on interpreting work in progress, articulating intentions and translating feedback into revisions. This technical report presents Critsly, an artefact-aware AI critique workspace, and StudioCrit, its architecture-studio research mode. Critsly combines a visual board, design-intention fields, guided reflection, perspective-based critique and action planning. StudioCrit adds studio/class organisation, role-based access, cognitive and architectural classification, educator analytics and exportable evidence. The report consolidates implementation and simulation evidence recorded in a research project submitted in July 2026. Three simulated studio scenarios yielded 109 classified evidence rows, including 85 assigned to higher-order Bloom categories. A separate rehearsal using 50 disposable learner accounts yielded 56 evidence rows, including 46 assigned to higher-order categories. A subsequent hardening rehearsal recorded 50 completed sessions, 50 successful board pulls and 50 denials of student access to analytics. These are software and synthetic-trace observations, not measurements of learning gains or human cognitive performance. Automated classifications remain provisional, and the source report does not establish classifier accuracy or inter-rater reliability. The contribution is an implemented critique-to-evidence workflow and a bounded account of its readiness for further controlled evaluation.

Comment
13 pages, 8 figures, 4 tables. Technical report adapted from an SMT 99.580 research project submitted on 23 July 2026. Documents StudioCrit engineering and simulation evidence; related Critsly demo: arXiv:2607.09673. No human-participant learning outcomes are reported. v2 adds the AIEDx grant acknowledgement for Critsly's development

Closest papers

Nearest neighbours by meaning across the whole index.

When AI Can Build It, What Should We Assess? Rethinking Evidence of Engineering Competence

cs.SE cs.HC ·2026-10-26 ·Ali Shakiba
similarity 0.67
Abstract · 11 pages, 1 figure, 4 tables

Generative AI can produce code, tests, documentation, and technical explanations that resemble artefacts traditionally used as evidence of student competence, weakening the inference from artefact quality to the engineering capability behind it. We report the redesign and first deployment of a 13-week, AI-permissive, software-intensive engineering course in which students may delegate substantial work to AI while retaining responsibility for planning, verification, review, integration, and explanation. The redesign couples an engineering accountability workflow, which structures human-AI work and preserves human decision authority, with an assessment evidence architecture comprising six complementary channels: product, process/origin, verification, explanation, reflection, and individual assurance. A collect--cross-check--escalate (CCE) reasoning logic connects these channels rather than treating any single artefact or trace as sufficient evidence of competence. By the end of Week 11, the deployment involved 87 students organised into 17 teams, with 957 individual weekly Git repositories and 34 group-project Git repositories supporting repeated assessment and engineering activity. The first deployment yielded four practical lessons: AI assistance changes where rubric criteria carry evidential weight; multiple evidence channels require deliberate cross-checking and meaningful complementarity; the evidential strength realised in practice may differ from that intended in the assessment design; and automation scales evidence collection more readily than interpretation. The contribution is a conceptually grounded, deployed design for making engineering responsibility and competence more visible when AI can perform substantial artefact production.

0.20

Making the Invisible Visible: A Framework for Reflective AI Use in Software Engineering Education

cs.SE cs.CYcs.HC ·2026-09-28 ·Ali Shakiba, Thomas Chaffey
publishedhighly cited similarity 0.66
Abstract · 9 pages, 2 figures, Accepted for publication in the Proceedings of the 37th Annual Conference of the Australasian Association for Engineerin…

Generative AI (GenAI) is increasingly embedded in software engineering education, supporting activities such as requirements development, design exploration, documentation, and prototyping. However, educators often have visibility only into final artefacts, with limited insight into how students evaluate, verify, and refine AI-generated outputs during the learning process. This creates challenges for assessing evaluative judgement and responsible AI-assisted practice. This paper introduces the AI Journal, a structured reflection framework designed to make student-GenAI interaction visible in first-year software engineering education. The framework combines execution tracking, which records prompts, outputs, intent, and interaction context, with cognitive auditing, which captures verification strategies, intervention decisions, confidence judgements, critical learning moments, and reflections on AI-supported work. Deployed in a first-semester software engineering course, the AI Journal enabled visibility into aspects of student learning not observable through artefact-based assessments alone. Preliminary observations suggested variation in verification practices, intervention strategies, and perceptions of AI-supported work. Critical learning moments frequently occurred when students evaluated contextual suitability, feasibility, and requirements alignment rather than identifying obvious errors. The AI Journal demonstrates a practical, lightweight, and model-agnostic approach for making AI-assisted learning processes visible. By foregrounding verification, intervention, and reflection, it shifts attention from product-focused assessment toward evaluative judgement and responsible AI-assisted practice.

0.51

Generative AI and Extended Reality in Collaborative Architectural Design Education: An Exploratory Studio Study

cs.HC cs.AI ·2026-09-11 ·Yao Xiao, Max Chen, Yichen Li, Nathaniel Powers, Maxwell Wiesenfeld +3 more
similarity 0.64
Abstract

Architectural design education relies heavily on visual ideation and representation to support collaborative learning in studio environments. Recent advances in generative artificial intelligence (GenAI) and extended reality (XR) offer new opportunities for rapid idea exploration and immersive spatial visualization. This exploratory mixed-methods classroom study investigated how GenAI-assisted multi-user XR influenced collaborative architectural conceptual design. We developed GenARch, a pipeline that integrates GenAI-based visual generation with collaborative XR environments, and deployed it in an undergraduate architectural design studio. Twenty-seven students formed seven self-selected design teams; four teams incorporated GenARch into their usual course workflow to support collaborative ideation and visualization, while three teams continued the same course workflow without GenARch. Pre- and post-intervention surveys assessed design self-efficacy, attitudes toward collaborative learning, and teamwork; a seven-member panel evaluated team design presentations; and GenARch teams participated in group interviews. The quantitative results showed larger relative declines in confidence and outcome expectancy for the GenARch condition and a positive difference-in-differences estimate for perceived conflict management, while panel-rated presentation outcomes were not significantly different between conditions. Interviews indicated complementary roles for the technologies: GenAI supported idea externalization and visual reference generation, whereas XR supported spatial, contextual, and scale-based evaluation. Students also reported challenges related to control, dimensional fidelity, shared attention, and motion comfort. These findings highlight both opportunities and limitations when GenAI and XR are incorporated into collaborative design education.

0.27

Critical Thinking with Generative AI: A Constraint-First Design Pilot of a Thinking-Partner Intervention

cs.HC ·2026-09-29 ·Fatima Tuz Zahra, Jiangen He, David M. Bowers, Wei Wang
similarity 0.63
Abstract · 53 pages, 5 figures

Generative AI (GenAI) tools entered higher education classrooms faster than the field was able to study their effects on learning. One concern is that GenAI may displace the critical thinking and AI literacy that students will need after graduation. This paper reports a Design-Based Research pilot of a GenAI-assisted critical thinking framework, in which ChatGPT was used as a thinking partner in an undergraduate research methods and statistics course during Spring 2025 (N = 14). The mixed-methods design combined pre- and post-intervention measures of statistical learning (AASCDM), AI literacy (MAILS), and critical thinking (WGCTA) with instructor field notes, student artifacts, and student-AI interaction logs. Pre-post tests showed gains on every AASCDM dimension and on eight of nine MAILS dimensions, while WGCTA percentiles did not change. Qualitative analysis identified four themes: the ways students positioned the LLM (as answer generator, validator, or co-thinker); the depth of student engagement (procedural vs. conceptual); occasional humanizing of the tool; and the role of curriculum design in shaping each of the prior three. Read together, the findings indicate that one semester of GenAI-assisted instruction can move domain learning and self-reported AI literacy but does not move standardized critical thinking, and that the modal student-LLM relationship is one of validation instead of dialogue. We end with design principles for the next iteration of the framework and implications for research on adaptive and personalized learning.

0.20

Judgment-Centred Software Engineering Education: A Post-Hype Review and Framework for AI-Augmented Learning

cs.SE cs.CY ·2026-09-24 ·Qusay H. Mahmoud
similarity 0.63
Abstract · 22 pages, 1 figure, 3 tables

Generative artificial intelligence has moved from a disruptive novelty to a recurring part of software-development and computing-education workflows, while software agents are beginning to act across repositories, command lines, browsers, tests, and other tools. The educational problem is no longer whether students should be allowed to generate code, but whether software-engineering (SE) programs can preserve and assess human understanding while preparing students to work responsibly with increasingly capable AI systems. This paper presents a structured integrative review of research and practice from 2023 through 23 September 2026, supplemented by established work on AI literacy, technical debt, and human-AI collaboration. The evidence supports a conditional conclusion: GenAI can improve access to explanations, feedback, practice, and short-term task completion, but learning outcomes depend on prior knowledge, scaffolding, verification, task design, and assessment. We examine student help-seeking and authorship, faculty assessment and policy demands, and the wider SE risks created when generated artifacts persist across repositories, teams, architectures, and deployed systems. We extend the concept of comprehension debt: the deferred learning and maintenance cost that arises when AI-assisted production outpaces a learner's or team's ability to explain, test, modify, and justify the resulting software. We then refine the AI-Augmented Software Engineering Education (AASEE) framework into five non-linear integration levels and four cross-cutting evidence obligations: explain, verify, modify, and account. The framework links delegation to evidence, governance, and recovery mechanisms appropriate to its consequences.

0.33

Integrating Flipped Learning and Generative AI for Practice-Based Design Education: Evidence from a Knit Yarn Design Course

cs.HC ·2026-09-16 ·Hong Qu, Zichao Ling, Yadie Yang
similarity 0.61
Abstract · 31 pages, 12 figures (including 3 appendix figures)

In practice-based design courses such as knit yarn design, students must turn visual ideas into feasible material outcomes. This is difficult because creative decisions are tied to yarn properties, stitch structures, machine operation, and limited opportunities for physical sampling. This study presents an integrated pedagogical framework that combines flipped learning, exemplar-based reference, GenAI-assisted visual prototyping, and studio feedback in an undergraduate knit yarn design course. The framework was implemented through a cross-device platform with pre-class micro-videos, formative checks, a curated gallery, and a GenAI-supported ideation module. An exploratory course-based evaluation compared a historical control cohort (N = 12) and an intervention cohort (N = 16), supplemented by questionnaire responses and brief interviews. The findings are interpreted as context-specific indicators rather than confirmatory causal evidence. Exploratory comparisons showed higher scores in creativity thinking, design skills, problem solving, and total course score in the intervention cohort. Student and instructor responses suggested that flipped learning supported studio readiness, while GenAI mainly supported early-stage visual exploration rather than precise technical guidance. Overall, the study offers a practice-based instructional framework for integrating flipped preparation, GenAI-assisted visual prototyping, and studio feedback in design education.

0.29

Dynamic Evidence Collection Ecosystem for Assessment Integrity and Authentic Competence

cs.HC cs.AI ·2026-08-17 ·Rajan Kadel, Bellal Hossain, Samar Shailendra, Bushra Naeem
hot direction similarity 0.61
Abstract · ISET 2026

Generative Artificial Intelligence (GenAI) can produce high-quality essays, code, and design artefacts, challenging the validity of conventional assessments that rely on single-point submissions and product-only grading. This paper proposes a design framework called "Dynamic Evidence Collection Ecosystem" that shifts assessment toward continuous, authentic, multi-source evidence of student learning over time. The framework collects process evidence through iterative artefacts, design logs, activity rounds, self-reflection, and peer collaboration, supported by an AI-enabled layer for learning analytics, formative feedback, and transparency. The approach is grounded in recent assessment-redesign scholarship in AI-rich contexts and aligned with contemporary views of authenticity in assessment. This paper builds on the hypothesis that academic integrity is strengthened when it is treated as an assessment design rather than as an AI detection problem. The tools have limitations and risks of use that carry academic penalties. This paper presents an implementation scenario to support institutional adoption.

0.33

Practical Implementation Report on Introducing Spec-Driven Development Using AI Agents in Software Development PBL

cs.SE ·2026-08-31 ·Hidetake Tanaka, Hiroshi Igaki, Kazumasa Shimari, Kiyoshi Honda, Naoki Fukuyasu
published similarity 0.61
Abstract · Accepted for publication in the 38th International Conference on Software Engineering Education and Training (CSEE&T 2026) : 18 pages, 1 tab…

In recent years, autonomous AI agents such as GitHub Copilot and Claude Code have been rapidly gaining popularity. This study reports on the practical implementation of Spec-Driven Development, a software development methodology premised on AI agents, within a Software Development Project-Based Learning (SDPBL) course for third-year undergraduate students. We defined a workflow consisting of four phases, namely investigation, planning, implementation, and review. We also established an environment tailored for the SDPBL course where AI agents generate documentation and code during each phase. We analyzed the results from three perspectives, namely students' subjective AI usage, implementation throughput, and code comprehension. The analysis reveals that AI usage patterns varied across development phases and teams. Moreover, while AI agent utilization increased implementation throughput, it also tended to encourage students to proceed with development without fully understanding the code. This study demonstrates that regular verification of code comprehension by instructors and appropriate feedback are essential for maintaining educational effectiveness when introducing SDD into SDPBL.

0.59