CS Paper Radar

Chance, Persistent Advantage, and the Generative-AI Era in Open-Source Package Careers

Hazem Ibrahim, Yasir Zaki

cs.SI cs.CY ·submitted 2026-09-09 04:03 UTC · v1
code/data

Abstract

Studies of careers in science, film, music, and books report a common pattern. When a person's most successful work arrives is close to a random draw over the works they produce. How large their successes tend to be, in contrast, follows a stable, person-specific factor. We test whether this pattern holds for open-source software careers and whether it changed when generative AI coding tools arrived. From the complete public record of GitHub push events (2015-2025), we reconstruct 102.2M career works by 6.15M contributors, and for the 908k contributors whose repositories publish packages, we measure each work's impact by how many downstream packages come to depend on it. First, we find that the timing of a career's biggest hit is close to a lottery over their works, as in science and the arts, with a small, replicable lean toward early career that grows as careers get longer. Second, some coders reliably produce higher-impact work than others, but this lasting personal factor accounts for only part of why impact persists (about a fifth in our primary specification); the rest behaves like momentum, success feeding on itself for a period of time. Third, within the same contributors, this structure did not change after ChatGPT's release. The stable factor's weight grew by about as much as it grew for an earlier cohort that simply aged, and subtracting the effect of aging from the effect of generative AI puts the shift at +0.03 (95% CI [-0.22, +0.23]), indistinguishable from zero. The success pattern documented in science and the arts therefore describes open-source careers too, and it shows no detectable break across the arrival of generative AI. These results have implications for how track records on open platforms should be read and on what to expect from generative AI for the careers built on them.

Comment
25 pages, 11 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Who Finishes the Job? A Study of Follow-Up Fixes and Commit Authorship on AI Coding Agent Pull Requests

cs.SE ·2026-09-22 ·Wannita Takerngsaksiri, Nhat Duong, Scott Barnett
code/data similarity 0.61
Abstract · 21 pages, 6 figures, 7 tables

AI coding agents now author a large share of pull requests (PRs) merged into popular open-source projects. A merged agent PR is usually considered finished work; yet, prior studies have reported issues in agent code after the merge (e.g., code smells and static-analysis issues). However, little is known about how often a merged agent PR is fixed afterward, and who actually authors the fixing. In this paper, we follow 6,774 merged agent PRs across five AI coding agents (OpenAI Codex, GitHub Copilot, Devin, Cursor, and Claude Code) from the AIDev-pop dataset (open-source repositories with at least 500 stars) into their follow-up fixes, against a baseline of 5,044 contemporaneous human PRs from the same repositories. We link each merge to its candidate fixes, verify every candidate with human annotators and an LLM judge that matches human-level agreement (binary Cohen's Kappa=0.78 against a human-human K=0.77, Direct-fix precision 90%), and attribute the fixing work at the PR and the commit level. Our findings show that (1) merged agent PRs attract verified fixes at 1.62 times the odds of merged human PRs in the same repositories over the same period of time; (2) 69.6% of verified fixes in agent merges come from the same agent; and (3) 76.4% of the verified fix PRs are agent-authored throughout all commits. These results show that agents currently largely finish their own job, but their merges still require fixing more often than human merges.

0.51

GitScholar: A Dataset for Predicting AI Research Impact from GitHub Engagement

cs.DL cs.AIcs.LGcs.SI ·2026-09-22 ·Emilien Guandalino, Lorenz K. Müller, Beatrice Alessandra Motetti, Konstantin Berestizshevsky, Lukas Cavigelli
datasetcode/data similarity 0.60
Abstract

With the rapid pace of AI research and the hundreds of daily new publications, staying up-to-date with the latest developments has become increasingly difficult. For researchers, quickly identifying impactful work is essential, yet manually reviewing each new publication is impractical. Automated impact prediction methods help address this challenge, usually by combining various information sources available, such as a paper's content or citation history. In this work, we propose using GitHub engagement as an additional source and demonstrate that it provides both a timely and accurate signal. To this end, we introduce GitScholar, a novel dataset that links GitHub activity from 444,000 repositories to over 558,000 AI arXiv papers. Our experiments show that GitHub reactions improve early prediction precision by up to 12% over a strong academic baseline. Additionally, we find that GitHub signal offers near-complete coverage of high-impact AI papers, and consistently correlates with future academic success. GitScholar is publicly available at https://huggingface.co/datasets/huawei-csl/GitScholar.

0.21

Engineering Signals of Human-AI Collaboration in the Agentic Coding Era: A Longitudinal Analysis of 33,228 Pull Requests from vLLM and SGLang with Implications for Biomedical AI Agents and Bioinformatics Pipeline Developmen

cs.SE cs.AIcs.ETcs.HC+1 ·2026-08-14 ·Jiada Li, Xuesong Ye, Olamide Olowoniyi
code/data similarity 0.59
Abstract · 24 pages, 9 Figures

The rapid adoption of AI coding assistants and autonomous agentic development systems has coincided with major changes in the pace and structure of open-source software engineering. Yet empirical longitudinal evidence of these changes at the team level remains limited. We present a descriptive longitudinal analysis of seven engineering metrics: pull request (PR) throughput, cycle time, contributor diversity, PR comment density, merge rate, new-author participation, and PR size. Metrics were computed from all merged PRs in two high-velocity AI infrastructure repositories, vLLM (February 2023-June 2026; 18,290 PRs) and SGLang (January 2024-June 2026; 14,938 PRs). We segment development into four eras aligned with major changes in AI-assisted software development and examine human- and bot-authored activities. Both projects show substantial increases in development velocity and AI-developer collaboration signals. PR throughput increased 21x in vLLM and 17.9x in SGLang, while bot-authored PRs accounted for less than 0.2% of this growth, indicating that the increase was overwhelmingly human-driven. In the latest era, median cycle time was 1.04 days for vLLM and 0.62 days for SGLang, while P90 cycle times reached 16.8 and 14.3 days, respectively. Monthly unique authors increased steadily in both projects, suggesting broader contributor participation. PR comment density increased 4.2x in vLLM and 3.8x in SGLang, with bot comments contributing an estimated 15-20% of the increase. In contrast, PR size remained relatively stable across eras. Overall, AI-assisted development is associated with higher throughput, broader contributor participation, and increased AI-developer collaboration signals in high-velocity open-source software development.

0.26

Skill Constellations: Tracing the Supply Chain of Agent Skills on GitHub

cs.SE cs.CRcs.SI ·2026-10-08 ·Fahd Seddik
similarity 0.58
Abstract · Project Website: https://fahdseddik.github.io/Skill-Constellations/

Agent skills are SKILL.md instructions and scripts that AI coding agents such as Claude Code and Codex run with the permissions of their user. Developers share skills by copying them between repositories, which makes them a software supply chain without a registry, versions or provenance. The origin of a copied skill, the reach of a security fix and the repositories that warrant review are therefore unknown. Studies that record which repositories hold a skill at a single point in time cannot reveal who copied it from whom. We contribute the first dated copy network of agent skills, built from the git history of every SKILL.md in GitSkills and covering 2,193,119 skill adoptions across GitHub, together with an interactive viewer. A few repositories are the source of almost all copies, and GitHub stars do not identify them. Skill copies almost never change with their source, and a fix at the source therefore rarely reaches them. We fit a model of which repositories others copy from and use it to rank repositories for audit. Reviewing the 100 repositories it ranks highest prevents 14.9% of later adoptions of high-risk skills, against 0.5% for the 100 most starred, which gives security engineers a short list to check before a skill spreads. Platforms should therefore distribute versioned references rather than copies. Project Website: https://fahdseddik.github.io/Skill-Constellations/

0.26

Code, Commits, and Complexity in the GenAI Era: A Longitudinal Study of Team-Based Capstone Projects

cs.CY cs.SE ·2026-11-05 ·Asma Shakil, Claus Brabrand, Paul Denny
published similarity 0.57
Abstract · Accepted to SIGCSE 2027

Generative AI tools are now widely available to students, but little is known about how team-based project codebases have changed since they appeared. This matters when project courses use repository activity as evidence of progress and learning. Existing studies have largely examined self-reports, controlled tasks, or short individual assignments, leaving open questions around how student codebases evolve in authentic project settings. We present a ten-semester analysis of N=260 capstone team repositories from 2021 to 2025, spanning pre- and post-AI cohorts. We examine three dimensions: code size (lines of code), code characteristics (comment density, error-handling density, and cyclomatic complexity), and development behaviour (commit volume, edit size, and commit frequency). We find that end-of-semester codebases roughly triple, from about 3.4 KLOC in 2021--2023 to about 10 KLOC in 2024--2025. Code characteristics show a clear decoupling: cyclomatic complexity and error-handling density increase, while comment density does not. Later cohorts commit far more often and in greater volume, but individual commit sizes are largely unchanged, suggesting a shift in development rhythm rather than edit granularity. Most changes appear from 2024 onwards and are not uniformly monotonic. We did not directly observe AI use, so we report these as changes that coincide with increased GenAI availability rather than as effects of AI use. Our findings highlight the need to reconsider how student-produced artefacts are used and assessed in project courses.

0.40

An Empirical Study of Maintainer-Identified AI Slop Pull Requests in Free and Open Source Software Projects

cs.SE ·2026-11-11 ·Hugo Raskin, Xavier Devroey, Benoît Vanderose
code/data similarity 0.57
Abstract · 12 pages

The FOSS ecosystem has always relied on outside contributions, notably through issues or Pull Requests (PRs), to sustain itself. With the advance of GenAI, the barrier to contributing to FOSS projects has never been lower, leading to the emergence of AI slop PRs: low-quality, low-value generated PRs. Past studies have examined AI slop PRs through theoretical, platform-level, or practitioner-account frames, but not at the project level. To this end, this study investigates the incidence of maintainer-identified AI slop PRs in FOSS projects by focusing on PRs flagged as slop by maintainers. Using a mixed-methods approach, we trace the origins of maintainer-identified AI slop PRs, quantify their volume in FOSS projects, describe their characteristics, and highlight maintainers' reasons for flagging or closing them. Our results show that the volume of maintainer-identified AI slop PRs increased in mid-2025, peaking in May 2026. These PRs are mostly additive, rarely merged, and typically closed within 3 days. The authors of these PRs disclosed the use of AI in 14% of cases, with 1% disclosing after a maintainer's request. Most PRs are closed or marked as slop without any reason provided by the maintainer. Based on our open coding, reasons range from concrete elements, like build failures, to ideological or intuitive (i.e., "looks AI to me"). This exploratory study captures a phenomenon in its early stage and highlights GenAI's potential impact on FOSS, emphasizing the pressure it puts on maintainers and the need to shift current open-source development practices.

0.45

The Customer Is the Builder: How AI Coding Agents Are Rewriting the Economics of Software

cs.SE cs.CYecon.GN ·2026-10-21 ·Ripon Chandra Malo, Zimeng Yu, Tong Qiu
benchmarkcode/datahot direction similarity 0.57
Abstract · 37 pages, 24 figures, 7 tables. Code and data: https://github.com/riponcm/The-Customer-Is-the-Builder. Archived release: https://doi.org/10.…

AI coding agents could change whether organizations buy software or build the functionality they need. We focus on organizations and professional buyers whose domain knowledge can help them specify and evaluate tools for their work. We define the software substitution frontier as the functions customers could build and maintain at an acceptable standard for less than buying them. Using public data, we compare growing capacity to build with indicators of market change. Since 2023, the length of benchmark tasks that leading agents can complete at 50 percent success, measured in human working time, has doubled about every 129 days. A broad search identified 575 potential open-source alternatives to selected commercial products, although some functional matches still require independent validation. The Craft suite produced 12 application repositories over 12.0 days of public development, but these remain early versions with unverified completeness and reliability. Market evidence is mixed: purchased software gained US investment share and tracked commercial installations grew. Yet 32 percent of respondents to an organizational survey reported forgoing at least one software purchase because their organization could build the functionality with coding agents. These self-reports do not establish the spending displaced or the reliability of the resulting tools. The gap motivates a warning: customers' capacity to build could affect individual buying decisions before a broad purchasing shift becomes visible. We examine this possibility using benchmark projections and observable market tests. The projections are an extrapolation, not a forecast of customer switching.

0.43

Sustainability of Open-Source Machine Learning Robustness Assessment Tools: A Repository Mining Study

cs.SE ·2026-08-28 ·Joshua Owotogbe, Indika Kumara, Willem-Jan van den Heuvel, Damian Tamburri
code/data similarity 0.57
Abstract · 32 pages. Preprint

Robustness evaluation is essential for deploying machine-learning (ML) systems in real-world settings, where models may face adversarial perturbations, distribution shifts, and other operational stressors. Many open-source tools, including Adversarial Robustness Toolbox, Foolbox, and Robustness Gym, support robustness testing and evaluation. However, little is known about how these tools are maintained, publicly engaged with, and sustained over time, even though practitioners may rely on them to select evaluation dependencies, reproduce robustness assessments, and provide evidence for AI assurance. We present an empirical study of the open-source robustness tooling ecosystem. Starting from a curated seed set derived from prior work, we systematically searched GitHub and identified 28 robustness-tool repositories. We analyzed repository artifacts to characterize observable community engagement, maintenance activity, and project longevity using established software-engineering metrics. Our results show that engagement and maintenance are unevenly distributed, with sustained activity concentrated in a small subset of repositories. At the data collection date of January 21, 2026, five repositories were classified as active, 22 as inactive, and one as archived. These findings highlight the need to treat robustness tools as evolving software systems.

0.35