CS Paper Radar

Navigating Complexity: A User-Centered Evaluation of Project Management Tools

Yasin Ghadimi Zaker, Hamid Reza Ghaeini

cs.SE cs.HC ·submitted 2026-11-06 10:19 UTC · v1

Abstract

In the age of digital transformation, companies are increasingly relying on specialized software to manage the complexity of projects. However, when selecting tools, features and costs often take precedence over user experience (UX), leading to low adoption rates, user frustration, and project delays. This study empirically evaluates the user experience of leading project management tools (Jira, Trello, Monday.com, Asana, and Microsoft Project) across various project scales using a mixed methods approach. The System Usability Scale (SUS) was used as the primary quantitative instrument, with data collected from 141 experienced professionals via online questionnaires. The results reveal a significant trade-off between functionality and usability. Trello achieved the highest SUS score (77.5), indicating a superior user experience for small projects and agile teams, while Microsoft Project scored the lowest (51.2), suggesting significant usability challenges despite its robust features. Jira demonstrated a balanced profile, offering powerful features with manageable complexity. Statistical analyses confirmed that the choice of the optimal tool is highly context dependent, with team size and project complexity influencing UX outcomes. The findings underscore that UX is not a secondary feature but a critical success factor for software acceptance and project effectiveness. This study provides an evidence-based framework for aligning tool selection with the project context, offers practitioners actionable insights, and identifies design opportunities for software developers.

Comment
11 Pages

Closest papers

Nearest neighbours by meaning across the whole index.

ADEMM: A Longitudinal Method for Monitoring Developer Efficiency in Industry

cs.SE ·2026-08-17 ·Danilo Ribeiro, Breno Alves, Gabriel Souza, César França, Alberto Souza
similarity 0.55
Abstract

Context: Developer efficiency is influenced by technical, organizational, cognitive, and communication-related factors. However, most studies rely on one-time assessments or fixed instruments, limiting the ability to monitor how barriers emerge and change over time, especially in consulting and professional education contexts. Objective: This study proposes and evaluates the Adaptive Developer Efficiency Monitoring Method (ADEMM), an adaptive longitudinal method for monitoring developer efficiency when the monitoring organization does not directly employ the developers. Method: Following Design Science Research and Action Design Research, we conducted a mixed-method longitudinal study with 27 software developers over twelve survey cycles. ADEMM was designed and refined through five iterative cycles, combining recurring surveys, 18 semi-structured interviews, and joint evaluation with a problem owner. Results: The study resulted in ADEMM, a method that supports continuous data collection, mixed-methods integration, and iterative redesign of monitoring instruments. The evaluation produced three design principles: prioritization with the problem owner based on actionability, combination of closed and open data collection, and adaptation of items based on low variance and emerging qualitative signals. Conclusions: ADEMM provides a transferable approach for adaptive longitudinal monitoring of developer efficiency. It helps balance comparability, contextual sensitivity, and practical utility in environments where organizations need to support developers without directly controlling their work contexts.

0.57

Software Project Management with LLM-Based Automation: Coordination, Validation, and Governance in Practice

cs.SE ·2026-09-02 ·Ronnie de Souza Santos, Cleyton Magalhaes, Italo Santos
similarity 0.55
Abstract

As software engineering has evolved, development environments have increasingly integrated automated support for tasks such as testing, analysis, and code generation, requiring project management to coordinate both human work and automated workflows. In this paper, we investigate how software project managers perceive changes in their work and learning demands associated with LLM-based automation. Motivated by a gap in software engineering research, which has largely focused on task-level and developer-centered uses of LLMs, we adopted an exploratory case study approach to capture managerial perspectives on automation in practice. Based on the experience of software project managers working in a large, multi-project software organization, our analysis indicates that LLM-based automation influences planning, estimation, coordination, monitoring, and governance activities rather than introducing new formal management practices. Participants described LLMs as becoming embedded in everyday project work, producing uneven effects on productivity, increasing the need for review and validation, and reducing visibility into task execution. These effects contribute to greater reliance on managerial judgment and coordination. Learning demands were perceived as experiential and incremental, centered on understanding LLM capabilities and limitations, critically assessing generated artifacts, and guiding responsible use within teams. The findings provide empirical evidence on how software project management work is adapted in contexts where LLM-based automation is integrated into ongoing software development practice.

0.36

Helpful but Fallible: Developer Experiences of AI Tools Under a Coordinated Industrial Roll-out

cs.SE ·2026-09-17 ·Andreas Bexell, Rushali Gupta, Lo Gullstrand Heander, Emma Söderberg, Per Runeson +3 more
similarity 0.54
Abstract

AI-enabled software development tools (AI-devtools) are being industrially adopted under strong expectations of productivity gains, yet developers' experiences of such roll-outs are underexplored. Organizations commit budgets, evaluate staff, and revise practice on a partial picture, since the evidence base is mainly tool evaluations, productivity metrics, and surveys, with few qualitative in-situ accounts of ongoing, coordinated roll-outs. We report a case study of a coordinated roll-out of AI-devtools at a large Swedish telecommunications company, investigating how developers experience the roll-out and how they anticipate their profession will change. We conducted semi-structured interviews with 12 software professionals across three sites, analyzed with process coding and thematic analysis, and interpreted through the extended Technology Acceptance Model (TAM2) as a post-hoc analytical lens. Our findings on use cases, productivity, frustrations, and tool limitations corroborate prior survey work. Beyond corroboration, the interviews surface a management-developer expectation gap that maps onto the TAM2 constructs of subjective norm and voluntariness, and show that participants weigh perceived risk heavily, a factor that TAM2 and similar acceptance models do not represent. AI-devtools emerge as helpful but fallible assistants whose value is shaped by organizational expectations, system scale, and developers' skills.

0.05

The Psychological Costs of Artificial Intelligence Adoption in Software Engineering

cs.SE cs.AI ·2026-09-03 ·Adam Alami, Elda Paja, Abhishek Tiwari
similarity 0.53
Abstract

Artificial intelligence (AI) is increasingly used to augment software engineering (SE) workflows. While code generation remains the main use case, organizations are actively seeking AI integration in other practices such as test cases generation and code reviews. Organizational AI adoption strategies seem to focus on tangible outcomes such as productivity. However, AI is a disruptive force, introduced into settings where role identity, team norms, and the sources of job satisfaction were well established before the recent advances in generative AI. Historically, technological disruptions have caused psychological and social strains in workplaces, ranging from anxiety and eroded meaning to deskilling and disrupted professional identities. The assumption that AI for SE is cost-free may not be accurate. Therefore, in this study we sought to understand the psychological costs software professionals experience during organizational AI adoption. We carried out a case study in a large software development services company, one year after the company launched its AI adoption. We collected qualitative data through meetings and semi-structured interviews (N = 21). We found that software professionals experience accountability anxiety, craft identity disruption, meaning and satisfaction erosion, cognitive and workload intensification, and uncertainty distress. Practitioners manage these costs through practices that restore control, mitigate them through protective and identity-preserving adaptations, or absorb them, carrying what neither can resolve. We contribute to AI-human collaboration in SE by repositioning AI adoption as a human transition, not only a technological and organizational one.

0.15

Designing Collaborative Virtual Environments for Software Engineering Education

cs.SE ·2026-11-10 ·Filipe Fernandes, Cláudia Werner
published similarity 0.53
Abstract · Submitted to Computers and Education: X Reality

Collaborative virtual environments offer opportunities to make software models available for shared inspection, but educational design and usability need to be examined separately from claims of learning effectiveness. This article presents MetaSEE, an approach connecting requirements guidance, a Web XR feature model, a reusable UML component, and a browser-based multiuser platform. We synthesize documented formative evaluations of the framework (16 participations), feature model (15), UML component (9), and platform (14). These counts represent evaluation participations; the same 12 developers took part in the first two evaluations. The principal platform study involved six teachers working individually and eight students working in four pairs through desktop or notebook computers, keyboard, and mouse. The documentary synthesis traces educational intentions to implemented features and reported difficulties. Participants identified potential value in shared modeling and simulation, alongside navigation, interaction, visualization, and distraction concerns. A numerical audit reconstructs a student System Usability Scale mean of 67.50 from published item frequencies, conflicting with the reported 79.38; an incomplete teacher item distribution prevents recovery of that group's overall mean. The contribution is a traceable account of educational design and formative limitations, with practical implications for onboarding, model readability, collaboration, and evaluation reporting. The evidence supports bounded feasibility and perceived usefulness, without establishing learning gains, sustained engagement, or adoption.

0.67

Human and LLM Facilitation in Collaborative Design Meetings: A Computational Behavioral Comparison

cs.LG ·2026-10-09 ·Mohammed Azeez Khan, Aaron Dsouza, Ashutosh Mishra, Ummul Faiz Z. Bibi, Neha K. Nair +1 more
similarity 0.53
Abstract · 17 pages, 7 figures, 6 tables

Large language models (LLMs) are increasingly being explored as facilitators of group discussion and collaborative work, but the behavioral differences between LLM-generated and human facilitation remain insufficiently characterized. We compare human Project Manager facilitation turns from scenario-based collaborative design meetings with Llama 3.1 8B Instruct responses across 199 paired contexts from the AMI Meeting Corpus. The analysis covers five confirmatory dimensions: contextual semantic distance, a lexical elaboration proxy, empathyand-hedge lexical density, topic divergence, and phase-vocabulary alignment. An exploratory directiveness measure failed domain validation and is not interpreted. The sampled human Project Manager turns were higher on contextual semantic distance (r = 0.644) and on empathyand-hedge lexical density (r = 0.203). The second contrast is carried mainly by hedge terms, so we read it as lexical hedging rather than psychological empathy. Llama-generated facilitation was higher on topic divergence (r = 0.703), the lexical elaboration proxy (r = 0.288; lengthconfounded), and instruction-conditioned phase-vocabulary alignment (r = 0.382). A secondary no-phase check left the Llama-higher phase contrast in place. The system prompt still requested a facilitation move, so that check is not evidence of spontaneous phase awareness. These differences should not be interpreted as a ranking of facilitation quality. The resulting behavioral profile provides a reference point for later comparisons of LLM facilitators, human facilitators, prompts, models, and collaborative settings.

0.29

Factors Impacting Developer Efficiency: Results from an Adaptive Longitudinal Study

cs.SE ·2026-08-17 ·Danilo Ribeiro, Breno Alves, Gabriel Souza, César França, Alberto Souza
similarity 0.52
Abstract

Context: Developer efficiency is driven by technical, organizational, and personal factors, yet few longitudinal studies explore how these factors evolve over time. Objective: This study investigates the primary factors hindering the perceived efficiency of developers in a consulting and professional development context, analyzing how these factors vary across recurring data collection cycles and how they are described qualitatively. Method: We conducted a mixed-methods longitudinal case study applying the Adaptive Developer Efficiency Monitoring Method (ADEMM) to 27 external software developers, combining twelve waves of periodic surveys with eighteen semi-structured interviews, analyzed through statistical and thematic analysis. Results: The most frequent bottlenecks were organizational dependencies and waiting for external validation, which stayed structurally stable, followed by technical knowledge gaps, which declined as developers adapted. A generative AI usage barrier emerged qualitatively nine waves into the study, was incorporated into the survey instrument, and became the most frequently coded interview theme. Interviews corroborated the quantitative findings, with insufficient requirements documentation and organizational dependencies as the most recurrent themes alongside AI-related challenges. Conclusions: Perceived developer efficiency is highly dynamic and cannot be accurately captured through a single cross-sectional measurement. Adaptive monitoring via ADEMM identified an emerging factor, generative AI usage barriers, that a fixed instrument would have missed, and informed a concrete organizational intervention during the study. For organizations managing external developers, actions should target external dependencies, communication channels, and developers' evolving use of AI tools.

0.22

Enjoying the Product but Not Joining the Builders: Investigating Why Students Don't Contribute to OSS

cs.SE ·2026-10-15 ·Italo Santos, Briana Lee, Derek Garcia, Yick Ching Wong, Anthony Peruma
surveycode/data similarity 0.52
Abstract

Open-source software (OSS) participation provides students with authentic exposure to collaborative, real-world software development, yet active contributions remain limited. This short paper presents an exploratory study of computing students' perceptions, knowledge, and experiences with OSS at the University of Hawai'i at Manoa. We administered a survey and analyzed 122 responses using both quantitative and qualitative methods. Students reported moderate OSS familiarity and frequent OSS use in coursework and personal projects, yet only 12.3% actively contributed and engagement with OSS communities was limited. Key barriers included skill gaps (72.89% of non-contributors), time constraints (59.81%), and codebase complexity (66.67% of contributors), often alongside imposter syndrome (36.45%). Students indicated that more time, mentorship, beginner-friendly issues, better onboarding/documentation, and curricular incentives (e.g., course credit) would most increase participation, while most reported their courses do not require or incentivize OSS contributions (58.20%). Grounded in a single institutional context, this study provides a nuanced understanding of how curriculum structure, faculty encouragement, and competing academic demands shape OSS participation. These insights inform the design of targeted curricular and institutional interventions aimed at supporting student onboarding and participation in OSS communities.

0.32