CS Paper Radar

The Vibe Paradox: What EEG Reveals About Your Brain When Vibe Coding

Olivia Schutz, Rudrajit Choudhuri, Yunhan Qiao, Fatima A. Moussaoui, Julia Russo, Zoe Steine-Hanson, Anita Sarma

cs.HC ·submitted 2026-11-01 17:46 UTC · v1

Abstract

Background: Much of the enthusiasm for vibe coding rests on two intuitive beliefs: (1) that developers stay invested even when they no longer write the code, and (2) that delegating more of the work to AI lowers their cognitive load. Objective: Our goal was to investigate whether developers' neurophysiological states and interactions with AI are consistent with these beliefs when they work with agentic vs. non-agentic AI on their own project vs. a legacy codebase. Method: We conducted an exploratory within-subjects crossover study with 11 participants across 22 sessions. Each participant used GitHub Copilot Chat in one session and Cursor Agents in the other, working on their own project and a legacy codebase, with tool-codebase pairings randomly assigned and session order counterbalanced. We recorded participants' EEG, on-screen activity, and prompts, and analyzed 7,924 EEG data points, 984 activities, and 141 prompts. Results: Our findings contradict both beliefs. With the agent, participants remained indifferent to task outcomes, even on their own projects, and cognitive load was higher than with non-agentic AI. Furthermore, their ways of interacting shifted: with non-agentic AI, participants asked how the code worked and read it to verify changes; with the agent, they described desired app behavior and checked results in the running app. Conclusion: Our study reveals a vibe paradox: delegating to an agent left participants motivationally disinterested, even on their own projects, without relieving cognitive load, while coinciding with increased trust in AI. The community should treat this paradox as a first-class concern if developers and their AI agents are to go far together, and not just fast.

Comment
12 pages, 5 figures, 1 table. Olivia Schutz and Rudrajit Choudhuri are co-first authors and have contributed equally

Closest papers

Nearest neighbours by meaning across the whole index.

Vibe Coding: Practice, Performance, Productivity, and Risk -A State-of-the-Art Review

cs.SE ·2026-08-20 ·Dominik L. Michels, Mutaz Abu Ghazaleh, Francois Lazzari, Nabil Kassem, Jonathan Klein
surveyposition similarity 0.70
Abstract

Vibe coding - AI-assisted software development in which the developer describes intent in natural language and validates results by running rather than reading the generated code - was named by Andrej Karpathy in February 2025 and produced its first body of empirical evidence within seventeen months. This state-of-the-art review assembles that evidence across a cross-disciplinary corpus spanning software engineering, human-computer interaction, labour economics, security research, governance, and education. We survey the model landscape, the tool ecosystem, and the performance record by task type, finding the early benchmarks saturated but task-level capability uneven: reliable code generation alongside weak fault detection and hard-to-audit documentation. The productivity record is at first contradictory: peer-reviewed field experiments report +26% more tasks per week, independent randomised trials measure a 19% slowdown, and team-level telemetry shows code-review time up +441%. We argue these readings are consistent once measurement method, scope, and time horizon are held constant, and identify six patterns behind the dispersion, among them effect-shrinkage under broader measurement, self-report diverging from independent measurement, output volume conflated with productivity, and bold claims walked back once tested over longer horizons. We further document security failures in deployed applications, code-quality degradation visible in large-scale code and developer telemetry, unsettled copyright exposure, and evidence of skill atrophy. The review closes with the open research questions and one falsifiable conjecture: that the gains are real on new code and shrink or reverse on mature codebases, which would account for most of the disagreement in the record.

0.20

Vibe-GUIDE: A Graph-based User Interface in IDEs for Oversight in Vibe Coding

cs.HC ·2026-09-20 ·Chifang Chou, Sam Yu-Te Lee, Rudrajit Choudhuri, Kwan-Liu Ma
similarity 0.64
Abstract · 14 pages, 3 figures, 4 tables

In agentic coding, developers shift from implementing changes themselves to specifying intent, evaluating the agent's work, and making approval decisions. However, delegating implementation can introduce cognitive debt that erodes project comprehension over time, constraining developers' ability to provide oversight. In this work, we investigate the role of persistent shared representations in supporting project comprehension and oversight of coding agents. We present Vibe-GUIDE, an agentic coding interface built around a structural, live, manipulable, and adaptive graph representation organized by functional modules. We evaluated our interface in a randomized between-subjects study comparing how 16 developers completed three cumulative coding tasks using our interface or a Chat-only baseline. We found that Vibe-GUIDE can support project comprehension and sustained task performance while keeping developers cognitively involved in oversight. These findings show how persistent shared representations can complement natural-language interaction and help developers maintain the understanding needed to oversee agent-generated changes as projects evolve.

0.42

The Vibe Shift in Software Engineering: Evaluating AI-Led Conversational Programming for Performance, Cognition, and Responsible Adoption

cs.SE cs.AIcs.ET ·2026-09-09 ·Sales G. Aribe, Louie Jay S. Labastida
published similarity 0.63
Abstract · 14 pages, 4 figures, 5 tables, Published by International Journal on Advanced Science, Engineering and Information Technology (IJASEIT)

This study evaluates Vibe Coding, an emerging AI-led conversational programming paradigm that enables developers to generate software through natural-language interaction with large language models. Using a mixed-methods design, the study assessed performance efficiency, cognitive implications, and responsible adoption in comparison with traditional and AI-assisted coding environments. Thirty participants, including professional developers and advanced computing students, completed equivalent programming tasks under three experimental conditions. Quantitative data were analyzed using descriptive statistics and repeated-measures ANOVA, while qualitative data were examined through thematic analysis. Results show that vibe coding significantly improved development efficiency, reducing task completion time by 27% compared with traditional coding and 12% compared with AI-assisted coding. However, these gains were accompanied by lower maintainability indices and higher security vulnerabilities, indicating trade-offs in software quality. Usability results yielded a good rating (SUS = 71.4), while cognitive workload remained moderate (NASA-TLX = 55.5), reflecting reduced syntactic effort but increased linguistic reasoning. Thematic analysis identified trust calibration, loss of control, cognitive adaptation, and prompt-engineering strategy as key constructs. Notably, perceived loss of control was associated with increased security risks due to reduced transparency and validation of AI-generated outputs. Based on these findings, the study proposes a three-pillar framework for responsible adoption: hybrid integration of human and AI capabilities, human oversight and transparent accountability, and context-aware deployment. Overall, vibe coding enhances productivity but requires critical oversight, reinforcing its role as a transformative yet transitional paradigm in software development.

0.19

Beyond the Prompt: Linking What Developers Ask, Do, and Understand with Coding Agents

cs.SE ·2026-09-27 ·Yunhan Qiao, Summit Haque, Christopher Hundhausen
similarity 0.63
Abstract

Coding agents can now change code for developers, who describe goals, supply context, and respond to the agent's work. Yet prompts, screen activity, and task success each tell only part of this story. We present Say, Do, Understand, an end-to-end workflow for analyzing what developers write to an agent, what they do while it works, and what they can explain afterwards. The workflow has five stages (Capture, Prepare, Analyze, Integrate, and Interpret) and three instruments: a prompt codebook, a scheme for coding screen-recorded activities and events, and separate rubrics for explaining the process and the solution. We applied it in an observational study of ten experienced developers who used GitHub Copilot on an unfamiliar codebase. Crossing task performance with understanding produced four personas. The two measures agreed for eight developers but split for two: one passed most tests but could not explain the solution, and another passed few tests but explained it well. In this sample, the personas that most often asked the agent to check its work spent the least time testing on their own. These patterns are descriptive and do not generalize beyond the sample. We recommend the workflow to computing educators, industry practitioners, and researchers to adapt and evaluate human--AI communication in software engineering.

0.19

Large-Scale Study of GitHub Copilot Trajectories

cs.SE ·2026-10-23 ·Izah Sohail, Yerdanat Kurmantayev, May Mahmoud, Sarah Nadi
similarity 0.62
Abstract

AI coding agents are increasingly integrated into software development workflows. However, most coding agent studies rely on benchmarks similar to SWE-bench, which assume autonomous agents operating in a single trajectory. This assumption does not mirror real-world usage, where developers interactively collaborate with agents. We address this gap by studying human-in-the-loop agent trajectories in the wild, focusing on 7,581 Copilot-authored PRs with publicly accessible session data. We find that 58% of Copilot PRs have no human intervention, and that when developers do intervene, collaboration is brief (median of 2 Human-AI loops). Merged PRs also tend to have fewer initial trajectory iterations than unmerged ones. We also find that the trajectory following the first human intervention exhibits a significant drop of 20.8 iterations and 8.4 min in duration. However, trajectories following later interventions show minimal iteration drops and no significant duration change, signifying that most of the work happens in the initial trajectory. Our qualitative analysis of illustrative PRs explains this pattern: many developer interventions are small, focused requests that Copilot completes in shorter later trajectories. We also illustrate that developers collaborate with Copilot in multiple ways, including delegated work through feedback as well as adding their own commits. However, we find that developers may need to guide Copilot through multiple (sometimes repeated) interventions in the PR, or even across multiple editions of the PR.

0.24

The Comprehension Gap in LLM-Assisted Software Development: An Experiment

cs.SE ·2026-11-18 ·Julian Oertel, Jil Klünder, Regina Hebig
similarity 0.62
Abstract

When LLM tools are pushed beyond their capabilities for software development tasks, developers must be able to step in. However, the comprehension that developers acquire during development might be negatively impacted due to excessive code generation. This could create a cognitive debt that needs to be resolved before tasks can be solved efficiently again. We explore how well developers understand code they generated by prompting in comparison to self-written and unseen code and how they reflect on their comprehension. We conduct a crossover experiment and interviews with 35 participants consisting of students, academic personnel, and software developers. After completing two programming tasks with or without LLM chat usage, respectively, participants are tasked with finding introduced mutations in the produced and additional unseen code. We use insights from interviews and screen recordings to supplement our results. Regarding the experiment, 36% of participants report a better understanding of self-written code compared to their generated code. On average, participants solved the comprehension task 20% faster for self-written code than for generated code and found 11% more mutations. Participants needed significantly less time to solve programming tasks when working with the LLM. However, participants with high reliance on the LLM chat and low task engagement showed significantly decreased comprehension. Our qualitative results show great variability in individual perspectives on comprehension differences and strategies to mitigate issues. Strategies for dealing with comprehension differences range from accepting comprehension differences to adapting their LLM interactions or avoiding the tools altogether. Our results warrant further investigation into cognitive debt in larger systems and developer practices to mitigate issues.

0.24

Comparing the Quality of Code Generated by Vibe Coding Tools

cs.SE ·2026-08-17 ·Gustavo da Mota, Kiev Gama
similarity 0.61
Abstract

The use of AI agents for automatic code generation has become increasingly common in software development. However, concerns remain about the quality of the generated code, including aspects of maintainability, readability, and long-term evolution. This study compares the structural quality of code produced by three widely adopted vibe coding tools --- Lovable, v0, and Replit --- starting from a single generation prompt. We generate three independent projects per tool, totalling nine web applications, and submit them to static analysis with SonarQube. We collect metrics such as the number of issues, severity distribution, estimated remediation effort, cyclomatic and cognitive complexity, and code duplication. Preliminary results show that the tools exhibit distinct qualitative profiles: Lovable concentrates issues of lower severity but presents a substantially higher density of code smells per KLOC, while v0 and Replit produce more code with more aggressive severity profiles. These findings suggest that choosing between vibe coding tools involves structural trade-offs that go beyond perceived productivity.

0.28

Beyond Productivity: Measuring Developers' Cognitive Load During GenAI-Supported Software Development

cs.SE cs.HC ·2026-09-29 ·Charlotte Brandebusemeyer, Daniela Gasser, Tobias Schimmer, Bert Arnrich
similarity 0.61
Abstract

Generative AI (GenAI) is changing software development workflows and how developers work. Industry evaluations of GenAI adoption often monitor productivity gains, usage, and output quality, but limited attention is paid to the interaction experience and cognitive load of the actual adopters and drivers of GenAI technology - the software developers. Understanding whether GenAI changes or shifts developers' cognitive demands during everyday development is important for a developer-centered evaluation of GenAI-supported software development. It can inform organizations in designing and evaluating effective AI-supported workflows. In this work, we study how GenAI use and task context relate to professional developers' perceived cognitive load and whether wearable-derived physiological characteristics provide additional information beyond this context. In a four-day industrial field study at two SAP sites, 21 developers documented their tasks, task duration, GenAI use, and perceived cognitive load while wearing an EmbracePlus wristband. The results show that perceived cognitive load is associated with both GenAI use and task context, while physiological measures provide only limited additional information. These findings suggest that developers' perceived cognitive load during GenAI-supported software development should be evaluated in relation to the concrete work context, with wearable physiological data used as complementary rather than standalone information.

0.12