CS Paper Radar

Cognitive Inclusivity in AI Coding Assistants: Evidence from GitHub Copilot

Grace Kerr, Aarushi Parasrampuria, Faith Culas, Kelly Blincoe

cs.SE ·submitted 2026-10-30 03:14 UTC · v1

Abstract

AI coding assistants are commonly used for daily software development tasks. While prior research has examined their impact on productivity and code quality, less is understood about how these tools support diverse ways of thinking. Evidence from non-AI software engineering tools suggests that mismatches between tool design and users' cognitive styles can introduce inclusivity barriers, but it remains unclear whether similar issues arise in AI coding assistants. We investigate how developers with different cognitive styles experience and interact with these tools and whether they exhibit cognitive inclusivity challenges. We conducted a controlled laboratory study with 20 student participants using GitHub Copilot to complete code generation, debugging, and refactoring tasks. Using think-aloud protocols, semi-structured interviews, and inductive thematic analysis, we identify patterns in user interaction. We then apply the GenderMag framework to examine how these patterns differ across cognitive styles. We identify five inclusivity issues concerning how Copilot explains suggestions, supports validation, facilitates learning, exposes its features, and influences the problem-solving process. Our findings suggest that common interaction paradigms in AI coding assistants can privilege certain cognitive styles while introducing additional cognitive effort for others. We derive design implications for more inclusive AI coding assistants.

Closest papers

Nearest neighbours by meaning across the whole index.

Beyond the Prompt: Linking What Developers Ask, Do, and Understand with Coding Agents

cs.SE ·2026-09-27 ·Yunhan Qiao, Summit Haque, Christopher Hundhausen
similarity 0.68
Abstract

Coding agents can now change code for developers, who describe goals, supply context, and respond to the agent's work. Yet prompts, screen activity, and task success each tell only part of this story. We present Say, Do, Understand, an end-to-end workflow for analyzing what developers write to an agent, what they do while it works, and what they can explain afterwards. The workflow has five stages (Capture, Prepare, Analyze, Integrate, and Interpret) and three instruments: a prompt codebook, a scheme for coding screen-recorded activities and events, and separate rubrics for explaining the process and the solution. We applied it in an observational study of ten experienced developers who used GitHub Copilot on an unfamiliar codebase. Crossing task performance with understanding produced four personas. The two measures agreed for eight developers but split for two: one passed most tests but could not explain the solution, and another passed few tests but explained it well. In this sample, the personas that most often asked the agent to check its work spent the least time testing on their own. These patterns are descriptive and do not generalize beyond the sample. We recommend the workflow to computing educators, industry practitioners, and researchers to adapt and evaluate human--AI communication in software engineering.

0.19

Practical Implementation Report on Introducing Spec-Driven Development Using AI Agents in Software Development PBL

cs.SE ·2026-08-31 ·Hidetake Tanaka, Hiroshi Igaki, Kazumasa Shimari, Kiyoshi Honda, Naoki Fukuyasu
published similarity 0.68
Abstract · Accepted for publication in the 38th International Conference on Software Engineering Education and Training (CSEE&T 2026) : 18 pages, 1 tab…

In recent years, autonomous AI agents such as GitHub Copilot and Claude Code have been rapidly gaining popularity. This study reports on the practical implementation of Spec-Driven Development, a software development methodology premised on AI agents, within a Software Development Project-Based Learning (SDPBL) course for third-year undergraduate students. We defined a workflow consisting of four phases, namely investigation, planning, implementation, and review. We also established an environment tailored for the SDPBL course where AI agents generate documentation and code during each phase. We analyzed the results from three perspectives, namely students' subjective AI usage, implementation throughput, and code comprehension. The analysis reveals that AI usage patterns varied across development phases and teams. Moreover, while AI agent utilization increased implementation throughput, it also tended to encourage students to proceed with development without fully understanding the code. This study demonstrates that regular verification of code comprehension by instructors and appropriate feedback are essential for maintaining educational effectiveness when introducing SDD into SDPBL.

0.59

How Developer Personality Shapes the Use of AI Coding Assistants

cs.AI cs.SE ·2026-11-16 ·Annemarie Wittig, Alina Mailach, Norbert Siegmund
similarity 0.67
Abstract

Developers differ in personality traits that shape how they approach software development tasks, yet AI coding assistants either behave uniformly or offer personalization features whose fit with individual developers must be found through trial and error. This is because it remains unclear how personality shapes developers' experiences with AI assistants, and thus which traits personalization should account for. In other words, we provide tools with no underlying theory on what works when and why. We take a first step towards such a theory with a systematic scoping review on how users' personality relates to their perception of LLM-based tools. We identified twelve studies through general search engines, major Software Engineering (SE) venues, AI tooling, and snowballing. They all reported associations between personality and perceptions, particularly perceived usefulness, yet none addressed this in a SE context. Building on these findings, we conducted an exploratory two-wave survey with 29 and 11 developers, collecting first empirical evidence of how developers' personality (measured using the Big Five Model) relates to the perception of AI coding assistants. Higher Extraversion was associated with more positive private-use experiences in both waves, while Agreeableness was associated with negative professional experiences in the first wave and positive ones in the second. These results provide first indications that the personality effects found in the literature extend to developers. Although further studies with bigger samples are needed to build a theory, our work takes the first step towards an empirical grounded theory of developer personality and AI personalization traits.

0.23

Feeling Easier, Working Harder: Cognitive Load in AI-Assisted Code Comprehension

cs.SE cs.AI ·2026-11-09 ·Maryangela White, Maximilian Rose, Matthew Buckman, Amy McDonnell, Francisco Ortega +1 more
top venue similarity 0.66
Abstract · Accepted to ICSE 2027

Software developers spend significant time reading and understanding code, increasingly supplemented by generative AI. While these tools aim to reduce the difficulty of reading code, interacting with them introduces a dual-task environment that may impose extraneous cognitive load that is challenging to capture. This study investigates how access to generative AI affects developers' real-time cognitive workload, code comprehension performance, and self-perceived code comprehension strategies. We conducted a 3x2 within-subjects study with 20 novice CS students reviewing Java code snippets of varying lengths with or without AI assistance, followed by an AI-unassisted comprehension quiz. Cognitive workload was continuously captured using a Detection-Response Task (tDRT) during both reading and quiz phases, and compared to post-quiz NASA-TLX and SEQ scores. AI assistance significantly increased objective cognitive workload, raising tDRT reaction times by 11.2%, yet participants' perceived workload remained unchanged. Code comprehension scores were high under both conditions, with no evidence that AI assistance improved comprehension at the difficulty levels evaluated; a ceiling effect limited power to detect benefits on harder tasks. Qualitatively, participants reported trusting the AI conditionally -verifying its output against their own reading for more complex snippets, but directly trusting it for simpler code- and, while many felt their reading process was unchanged, a notable subset shifted toward skimming or reading the AI's explanations in place of the code itself. These findings reveal that developers do not accurately perceive the cognitive resources they expend when managing AI tools. This study provides evidence of added cognitive effort associated with using generative AI, pointing towards a need to account for this dual-task penalty in software engineering tools and education.

0.62

Vibe-GUIDE: A Graph-based User Interface in IDEs for Oversight in Vibe Coding

cs.HC ·2026-09-20 ·Chifang Chou, Sam Yu-Te Lee, Rudrajit Choudhuri, Kwan-Liu Ma
similarity 0.66
Abstract · 14 pages, 3 figures, 4 tables

In agentic coding, developers shift from implementing changes themselves to specifying intent, evaluating the agent's work, and making approval decisions. However, delegating implementation can introduce cognitive debt that erodes project comprehension over time, constraining developers' ability to provide oversight. In this work, we investigate the role of persistent shared representations in supporting project comprehension and oversight of coding agents. We present Vibe-GUIDE, an agentic coding interface built around a structural, live, manipulable, and adaptive graph representation organized by functional modules. We evaluated our interface in a randomized between-subjects study comparing how 16 developers completed three cumulative coding tasks using our interface or a Chat-only baseline. We found that Vibe-GUIDE can support project comprehension and sustained task performance while keeping developers cognitively involved in oversight. These findings show how persistent shared representations can complement natural-language interaction and help developers maintain the understanding needed to oversee agent-generated changes as projects evolve.

0.42

Beyond the Traceback: Using LLMs for Adaptive Explanations of Programming Errors

cs.SE cs.HC ·2026-08-21 ·Alexandru-Radu Moraru, Shreyan Biswas, Ujwal Gadiraju
similarity 0.65
Abstract

Programming error messages are critical for software development, yet they remain difficult for novice programmers to interpret. While Large Language Models (LLMs) can rewrite these errors into clearer explanations, it remains unclear whether increased readability improves objective debugging performance or how explanation styles should align with programmer skill. We present a multi-stage crowdsourced study N=103 evaluating skill-targeted, LLM-generated Python error messages. Using a custom proficiency assessment, we categorized participants by skill level and tested standard interpreter messages against two LLM-generated styles: pragmatic (action-oriented) and contingent (scaffolded explanations). We measured both objective debugging metrics (fix rate, attempts, time-to-fix) and subjective perceptions (readability, cognitive load, tone). Our results show that while LLM-rewritten messages significantly improved subjective evaluations, with pragmatic messages rated as clearer and less cognitively demanding, these perceived gains did not translate into statistically significant improvements in objective debugging performance. This highlights a critical human-AI complementarity gap: explanations that feel better to users do not necessarily make them more effective debuggers. We discuss design implications for adaptive AI feedback systems, arguing that future tools should pivot from static skill-targeted rewriting toward dynamic adjustments based on a user's real-time repair trajectory.

0.20

SWE-Journey: Towards More Realistic Evaluation of Coding Assistants through Long-Horizon, Multi-Turn Interaction

cs.CL cs.AIcs.MAcs.SE ·2026-10-08 ·Hexuan Deng, Yue Wang, Wenyu Jiang, Cheng Yang, Haolin Yang +11 more
benchmarkhot direction similarity 0.65
Abstract

Coding assistants such as Claude Code and Codex have become a major application of LLM agents, yet existing benchmarks remain far from real-world use, particularly in task horizon and interaction length. Code assistants require completing long chains of development work in continuously evolving repositories, while repeatedly clarifying requirements and adapting implementations through multi-turn interaction. To address these gaps, we introduce SWE-Journey, a benchmark for more realistic evaluation of coding assistants. To address the task-horizon gap, we propose a weak-to-strong synthesis pipeline that automatically constructs long-horizon coding tasks. To address the interaction gap, we mine four representative user personas from real interaction data and build a user-simulation agent to reproduce realistic code-assistance interactions. On average, models pass over 75% of tests for requested functionality with software architects, but fewer than 25% with non-coders. These results show that current coding assistants still fall short of enabling reliable coding for non-coders. We further analyze the reasons for this gap and identify asking right, finding right, and fixing right as key capabilities during interaction.

0.23

Developers as Managers of AI Agents: A Workplace Shadowing Study

cs.SE ·2026-10-24 ·Yi-Hung Chou, June Hyung Kim, Vasisht Kartik, Victoria Jackson, Tobias Schimmer +2 more
similarity 0.65
Abstract

As AI coding agents become more autonomous, developers increasingly direct, coordinate, and check their work. Yet little is known about these practices in everyday professional development. We shadowed 11 professional developers across approximately 15 hours of work, coded their activities and task transitions, analyzed 255 submitted prompts, and conducted 4.6 hours of follow-up interviews. Developers spent an average of 33.4% of observed time formulating prompts or checking AI responses, with task-switching rates ranging from 0 to 101.6 switches per hour. Beyond producing artifacts, prompts supported understanding, requirements clarification, agent configuration, and coordination. We identified three recurring patterns: \textit{Supervisor}, involving direct artifact inspection; \textit{Delegator}, involving dialogue-based coordination and delegated implementation and checking; and \textit{Learner}, involving agents primarily to build understanding. Developers sought technical information from agents and organizational context, authority, and alignment from colleagues. Our findings suggest future studies of how developers manage attention across concurrent agent tasks and tools that support verification, coordination, and learning.

0.18