CS Paper Radar

Design, development, and preliminary validity and reliability evidence of the Software Engineering Self-Efficacy Scale (SESES)

Albert D. Ritzhaupt, Jhonathan Sora-Cardenas, Priya Ganapathy Prasad, Kai A. Hackney, Neha Rani, Pedro Guillermo Feijóo-García

cs.SE ·submitted 2026-09-24 05:57 UTC · v1

Abstract

The purpose of this research is to design, develop, implement, and provide preliminary validity and reliability evidence of the Software Engineering Self-Efficacy Scale (SESES). Framed by a conceptual framework using guidance in software engineering curriculum and concepts along with the notion of self-efficacy, we generated an initial item pool of n = 87 items to operationalize and measure software engineering self-efficacy among undergraduate computing students. The conceptual framework traces five dimensions: 1) Requirements Engineering, 2) Teamwork and Collaboration, 3) Software Quality Management, 4) Software Design and Architecture, and 5) Software Agile Methodologies. We pilot tested the SESES with n = 527 undergraduate computing students who had completed a software engineering course in the current semester or a previous academic semester. We employed Exploratory Factor Analysis (EFA) with the Principal Axis Factoring method and an oblique (Promax) rotation to examine the underlying structure of the SESES, resulting in the same five internally consistent latent constructs in the conceptual framework with minimal cross-loading and a simple structure in the pattern matrix, explaining approximately 57% of the variability in these data. Our findings suggest that software engineering self-efficacy is a multidimensional construct of five theorized and correlated, yet distinct latent factors. We unpack the limitations and delimitations of the research while exploring undergraduate computing students' software engineering self-efficacy using necessary domain-specific measurements.

Closest papers

Nearest neighbours by meaning across the whole index.

Assessing the Construct Validity of Object-Oriented, Class-Level Code Quality Metrics

cs.SE ·2026-09-17 ·Hera Arif, Miikka Kuutila, Paul Ralph
top venuecode/data similarity 0.58
Abstract · Accepted to the ESEIW 2026 ESEM - Technical Track

Background: Code quality metrics are intended to measure latent properties of software source code. Although numerous code metrics have been proposed and used, their construct validity is rarely evaluated. Thus, the extent to which code metrics actually measure what they claim to measure is often unclear. Aim: Drawing from modern measurement theory, we investigate the construct validity of common class-level, object-oriented code quality metrics by identifying their factor structure using Exploratory Factor Analysis (EFA). The metrics were extracted from the Apache Maven project by three software tools: Designite, JHawk, and Understand. The factor structure was later verified using Confirmatory Factor Analysis (CFA) on 22 randomly selected open source projects meeting a predetermined eligibility criteria. Results: 24 code quality metrics that correspond to six constructs: Cohesion, In-Coupling, Out-Coupling, Size, Sub-Inheritance (related to subclasses), and Sup-Inheritance (related to superclasses) were revealed in the underlying factor structure. Ten metrics did not correspond to any known dimension of software quality and were removed in the EFA. Ten additional metrics exhibited low loadings in the CFA, suggesting their removal from the final measurement model. Size, Cohesion, Inheritance, and Coupling were the constructs retained, with subcategories identified for Inheritance and Coupling. Conclusions: Our results strongly support the construct validity of 24 code quality metrics. Coupling and Inheritance are revealed as multidimensional constructs, since they require measuring two different concepts, revealed as sub-categories in our analysis, and Complexity may be better explored in a multilevel model. Overall, our study demonstrates the value of applying modern measurement theory and latent variable modeling in validating software code quality metrics.

0.70

Measuring undergraduate students' reliance on generative AI during problem-solving: Scale development and validation

cs.HC ·2026-10-21 ·Chenyu Hou, Gaoxia Zhu, Vidya Sudarshan, Fun Siong Lim, Yew Soon Ong
publishedhighly cited similarity 0.55
Abstract · If you are interested in accessing the full scale and instructions, please fill in this form: https://forms.gle/xbvg9CF84o3AW1SW7

Reliance on AI describes the behavioral patterns of when and how individuals depend on AI suggestions, and appropriate reliance patterns are necessary to achieve effective human-AI collaboration. Traditional measures often link reliance to decision-making outcomes, which may not be suitable for complex problem-solving tasks where outcomes are not binary (i.e., correct or incorrect) or immediately clear. Therefore, this study aims to develop a scale to measure undergraduate students' behaviors of using Generative AI during problem-solving tasks without directly linking them to specific outcomes. We conducted an exploratory factor analysis on 800 responses collected after students finished one problem-solving activity, which revealed four distinct factors: reflective use, cautious use, thoughtless use, and collaborative use. The overall scale has reached sufficient internal reliability (Cronbach's alpha = .84). Two confirmatory factor analyses (CFAs) were conducted to validate the factors using the remaining 730 responses from this activity and 1,173 responses from another problem-solving activity. CFA indices showed adequate model fit for data from both problem-solving tasks, suggesting that the scale can be applied to various human-AI problem-solving tasks. This study offers a validated scale to measure students' reliance behaviors in different human-AI problem-solving activities and provides implications for educators to responsively integrate Generative AI in higher education.

0.77

Psychological Safety in Software Engineering Teams: A Systematic Mapping Study of Team Processes and Performance

cs.SE ·2026-10-06 ·Adnan Manzoor, Muhammad Azeem Akbar, Jussi Kasurinen, Muhammad Hamza, Aakash Ahmad +1 more
similarity 0.52
Abstract

Context: Psychological safety is an important human and behavioral factor in software development teams, supporting collaboration, knowledge sharing, interpersonal risk-taking, and learning. However, evidence in software engineering remains fragmented across team processes, performance outcomes, contextual conditions, barriers. Objective: This study maps literature on psychological safety in software engineering, examining its conceptualization, associations with team performance and processes, contextual influences, barriers, and strengthening strategies. Method: We conducted a systematic mapping study following established software engineering guidelines. We analyzed 112 primary studies published between 2006 and June 2026 using descriptive statistics and thematic analysis across six research questions. Results: Psychological safety emerges as a multidimensional, context-dependent, team-level construct characterized by interpersonal risk-taking, open communication, collaboration, learning, trust, and supportive interaction. It is predominantly associated with productivity, software quality, innovation, team satisfaction, collaboration, communication, and decision-making. These relationships are shaped by leadership, organizational culture, work environment, trust, and sociocultural and individual conditions. Recurring barriers include communication problems, organizational constraints, leadership issues, diversity and inclusion challenges, and workload pressures. Although diverse practices are reported, evidence of their effectiveness and long-term sustainability remains limited. Conclusion: Psychological safety is a multidimensional, context-dependent team-level construct associated with team interaction and performance. This study highlights the need for longitudinal, context-sensitive research on psychological safety interventions in software development environments.

0.30

Designing Collaborative Virtual Environments for Software Engineering Education

cs.SE ·2026-11-10 ·Filipe Fernandes, Cláudia Werner
published similarity 0.52
Abstract · Submitted to Computers and Education: X Reality

Collaborative virtual environments offer opportunities to make software models available for shared inspection, but educational design and usability need to be examined separately from claims of learning effectiveness. This article presents MetaSEE, an approach connecting requirements guidance, a Web XR feature model, a reusable UML component, and a browser-based multiuser platform. We synthesize documented formative evaluations of the framework (16 participations), feature model (15), UML component (9), and platform (14). These counts represent evaluation participations; the same 12 developers took part in the first two evaluations. The principal platform study involved six teachers working individually and eight students working in four pairs through desktop or notebook computers, keyboard, and mouse. The documentary synthesis traces educational intentions to implemented features and reported difficulties. Participants identified potential value in shared modeling and simulation, alongside navigation, interaction, visualization, and distraction concerns. A numerical audit reconstructs a student System Usability Scale mean of 67.50 from published item frequencies, conflicting with the reported 79.38; an incomplete teacher item distribution prevents recovery of that group's overall mean. The contribution is a traceable account of educational design and formative limitations, with practical implications for onboarding, model readability, collaboration, and evaluation reporting. The evidence supports bounded feasibility and perceived usefulness, without establishing learning gains, sustained engagement, or adoption.

0.67

Beyond Correctness: How Novice Software Engineering Students Exercise Oversight of Generative AI

cs.SE cs.HC ·2026-10-23 ·Ali Shakiba, Thomas Chaffey
similarity 0.51
Abstract · 10 pages, 3 figures, 2 tables

Generative AI (GenAI) is shifting software engineering students' work from producing artefacts towards evaluating and supervising AI-generated outputs. However, little is known about what characteristics students identify in outputs when reporting interventions, or how they exercise oversight during authentic software engineering work. We analysed 962 student-AI interaction turns documented by 145 first-year software engineering students through structured AI Journals, including 172 student-declared interventions. Using a multi-dimensional coding framework, we characterised the output characteristics identified in these reported interventions, students' reasons for intervening, the actions they took, and how those actions were enacted. Fitness problems were identified in 74.4% of interventions, compared with 16.9% involving correctness problems, indicating that oversight more often concerned whether plausible AI outputs were suitable for the engineering task than whether they contained explicit errors. Among classifiable interventions, students most frequently modified generated outputs (51.0%) or refined their prompts (28.2%), with both response types commonly associated with engineering fit. Within-student analysis further showed that outputs perceived as containing minor flaws were associated with lower evaluation confidence than outputs perceived as perfect. Taken together, these findings characterise novice AI oversight as a process that extends beyond error detection to encompass both judgement of fitness for purpose and active intervention in AI-assisted engineering work.

0.49

Learner-Centered Design of Educational Tools for Cross-Expertise Technical Communication in Computing Contexts

cs.HC ·2026-09-20 ·Jinyoung Hur, Yoshee Jain, Yuxuan Chen, Ge Wang, Kathryn Cunningham
similarity 0.51
Abstract · 19 pages, 3 figures, 3 tables

Cross-expertise technical communication (i.e., communicating about computing topics across levels of computing expertise) is important for workplace collaboration. However, students are underprepared because computing education rarely explicitly teaches communication, and available practice tends to occur among peers with similar expertise. We take a learner-centered design approach to inform technologies for developing these skills. We interviewed 12 industry professionals and conducted focus group and co-design sessions with 14 college students preparing for computing-related roles. Professionals emphasized adapting communication, negotiating expectations, building shared understanding, and using multiple modalities and artifacts. Students anticipated challenges adapting communication, navigating unfamiliar workplace dynamics, and communicating under fear of judgment, and valued rich, realistic practice environments that preserved their psychological safety. Synthesizing professional and learner perspectives with educational theory, we conclude that building self-efficacy through authentic mastery experiences and changing attitudes related to help-seeking are critical elements for educational technology design in this context.

0.31

Surrendering and Reclaiming Judgment: How Software Engineering Students Experience AI Reliance

cs.SE ·2026-11-09 ·Nathan Cassee, Umit Akirmak, Luiz Guerra, Margaret-Anne Storey, Neil Ernst
similarity 0.50
Abstract · Submitted to TOSEM

Software architecture is one of the skills novice developers must master to succeed in industry, and as AI disrupts software development, it is also transforming how students learn architecture. Existing work on how students and novices work with AI has mostly focused on small programming exercises or on students' perceptions of AI, so little is known about how students experience AI as they learn software architecture in larger team projects. We report on an exploratory case study of 43 senior software engineering students who used AI throughout a semester-long, team-based software architecture project. We combined surveys with focus groups and analyzed the focus group transcripts and the open survey responses through Braun and Clarke's reflexive thematic analysis. We constructed two themes. The first, Surrendering Judgment to AI, traces how participants' reliance on AI deepened as they used it to architect and implement their systems, until they surrendered their judgment and became stuck in a codebase they had created but could not understand, which made them rely on AI even more. Participants described feeling ashamed, frustrated, and emotionally disconnected from their work, a toll that, in our reading, left them with even less reason to break their reliance. The second, Applying Human Judgment, centers on participants who kept or reclaimed their judgment: the knowledge and experience it rested on, and the strategies they used to apply it. Participants' accounts suggest that relaxed deadlines and a lecture and lab on working with AI gave them room to acquire knowledge and reclaim judgment. Given the expectations placed on junior developers and the widespread adoption of AI, we believe these first-hand accounts can help software engineering education and industry adopt AI in ways that preserve novice developers' judgment, ownership, and motivation.

0.28

Exploring Students' Insights on Creating Videos to Learn Software Design and Engineering

cs.CY ·2026-11-17 ·Pedro G. Feijóo-García, John D. Centner, Lucas G. Zangari, Chase G. Wrenn, Neha Rani +2 more
published similarity 0.50
Abstract · This work was presented at 2025 ASEE/IEEE Frontiers in Education (FIE)

This research-to-practice study examines the use of student-created instructional videos as an active learning strategy in a large undergraduate Software Design and Engineering (SWE) course. Supporting engagement and understanding in large-scale computer science (CS) courses remains a challenge, as instruction often relies on automated activities and summative assessments that prioritize grading over meaningful learning. This can make the learning experience feel impersonal, leading to frustration, anxiety, and confusion, especially when students face complex topics or new programming tools. To address these issues, we implemented a learner-centered strategy grounded in the principle of ``learning by teaching," where students created instructional videos to explain course concepts in their own words. This approach allowed students to take ownership of their learning, adapting the material to fit their individual perspectives and reinforcing their understanding through explanation. We report on students' reflections from a large introductory SWE course (over 700 students), analyzing their open-ended feedback on the benefits and drawbacks of the strategy. Students highlighted increased engagement and learning, and many appreciated the flexibility of the video-based format compared to traditional multiple-choice exams, which they found more restrictive. However, students also noted that the video creation process was time-consuming. This study contributes to the growing conversation on active learning in computing and engineering education by showcasing the potential of student-created instructional videos to enhance student agency, motivation, and conceptual understanding in large-scale learning environments.

0.26