CS Paper Radar

Understanding Maintenance and Support in a Community-Driven Scientific Workflow Ecosystem: A Cross-Space Study of Galaxy

Khairul Alam, Kowsik Roy, Md Shamimur Rahman, Banani Roy

cs.SE ·submitted 2026-09-22 02:32 UTC · v1

Abstract

Galaxy is a widely used, community-driven scientific workflow system whose sustainability depends on continuous maintenance across its software, tools, workflows, infrastructure, documentation, and user-support ecosystem. However, maintenance knowledge in Galaxy is distributed across development and community-support spaces, making it difficult to understand what is maintained, how maintenance artifacts are resolved, and how user-facing concerns connect to repository-level development. We conduct a large-scale empirical study of Galaxy using 11,762 GitHub issues, 52,203 pull requests, and 6,235 Community Forum discussions. We characterize maintenance and support concerns, examine factors associated with resolution outcomes and resolution time, and investigate explicit and candidate connections among maintenance artifacts across these spaces. Using BERTopic modeling, we identify nine issue topics, 14 pull-request topics, and 14 forum topics, revealing a maintenance landscape spanning workflow execution, data management, tools and dependencies, infrastructure, testing, scientific resources, documentation, and user support. Resolution analyses show that coordination, diagnostic, contributor, automation, and engagement characteristics exhibit different associations with whether artifacts are resolved and how quickly resolution occurs. We further find limited explicit traceability between development and support spaces: 97.77\% of 16,426 resolved explicit relationships occur within GitHub, while only 294 connect GitHub artifacts with Community Forum discussions, despite additional semantic and technical relatedness across these spaces. Together, these findings characterize Galaxy maintenance as a distributed ecosystem-level process and identify opportunities to improve diagnostic reporting, lifecycle-aware triage, cross-space traceability, and the reuse of community-support knowledge.

Comment
39

Closest papers

Nearest neighbours by meaning across the whole index.

SciWorkBench: A Benchmark for Real Scientific Workflows

cs.AI ·2026-11-17 ·Yuchen Zhang, Taolin Han, Jinghang Wang, Jinxin Wang, Yun Wu +17 more
benchmark similarity 0.64
Abstract

Language models are increasingly being used as agents capable of planning and executing projects, as well as refining their actions through feedback. This shift raises a question: can current agents complete projects like real scientists? To examine this question, we introduce SciWorkBench, a benchmark for evaluating agents' abilities on real science tasks across biology, chemistry, materials science, and physics. SciWorkBench is built from peer-reviewed literature or expert-contributed research scenarios. We also propose a unified scientific analysis framework validated by domain experts, which can categorize most tasks in SciWorkBench. Each task covers one or more stages of this framework. We evaluate up-to-date models within the Claude Code and Codex agent frameworks under a unified execution and grading protocol. The best-performing agent achieves a mean score of 57.38, indicating that current agents still struggle with realistic scientific workflows. Our failure analysis identifies three recurring weaknesses at the workflow level: difficulty translating scientific knowledge into task-specific procedures, weak integration of evidence across different modalities, and limited capacity for scientific self-correction. These findings highlight the importance of workflow-based evaluation for measuring progress toward reliable AI systems for scientific research.

0.11

The Other Half of Workflow Portability: Evidence-Backed HPC Site Profiles with Agentic Discovery

cs.DC cs.IR ·2026-10-01 ·Md Saiful Islam, Douglas Thain
published similarity 0.62
Abstract · Accepted to the 21st Workshop on Workflows in Support of Large-Scale Science (WORKS 2026), held with SC26, Chicago, IL, USA. 8 pages, 7 figu…

Moving a workflow developed and tested at one HPC site to another rarely succeeds without some amount of trial and error. Package managers rebuild software environments, containers ship whole filesystems, and workflow specifications such as backpacks package a workflow with its software, data, and resource requirements. These approaches address one half of workflow portability: what a workflow needs. But none describes how a given HPC site must be used, and that missing half is why even a portable workflow requires manual adjustment at each new site. That gap includes the site's resource shape, storage configuration, network permissions, and operating policies. This information may be explicit in the batch system, hidden in the prose of documentation, or buried deep within a router's configuration, making it difficult for an automated deployment tool to turn site knowledge into useful deployment decisions. We propose the HPC site profile, a structured, evidence-backed document that makes this knowledge actionable. We automatically construct it in three steps that mirror where the information lives: measuring the login node, extracting typed fields from documentation with a bounded language-model agent, and submitting pilot jobs for eligible unresolved fields. Every field is verified against its evidence or discarded, so a rule, not the model, decides what enters the profile. The profile then preflights a workflow into an execution plan or an early, explainable failure. We build profiles at Purdue Anvil, TACC Stampede3, and Notre Dame CRC and present a case study of preflighting a real workflow.

0.44

Missing Security-Fix Backporting for GitHub Actions Workflows

cs.SE cs.CR ·2026-11-17 ·Chengyan Ma, Jieke Shi, Ruidong Han, Weifeng Sun, Yuqing Niu +3 more
top venue similarity 0.62
Abstract · Accepted by the 49th IEEE/ACM International Conference on Software Engineering (ICSE 2027)

GitHub natively supports workflow automation through GitHub Actions, now the dominant CI/CD tool on the platform. Yet keeping these workflows secure remains an ongoing burden: maintainers often patch a vulnerability on the default branch but leave the same flaw on the release and maintenance branches that downstream consumers continue to build from. Carrying a known fix onto these divergent branches, called backporting, is rarely done, and this paper presents the first large-scale empirical study quantifying the resulting gap. From a public corpus of GitHub Actions workflow histories, we mine 1,903,191 workflow-modifying commits, scan each before and after with the static scanner zizmor to identify 69,865 security fixes across 24,238 projects, and audit every release branch to check whether each fix was carried over. The gap is pervasive: 224,392 of 474,490 audited release branches remain vulnerable, and deliberate backporting is rare and slow, with only 19,476 of 78,907 already-fixed branches confirmed as plausible backports at a median lag of 243 days. We thus design WorkflowBP, the first technique for automated security-fix backporting of GitHub Actions workflows. WorkflowBP takes a neuro-symbolic approach: it compiles a default-branch fix into a domain-specific representation called the Workflow Semantic Patch (WSP) and applies it to each divergent branch by matching jobs and steps based on their content rather than their position, while an LLM synthesizes the WSP for the most divergent cases the deterministic engine cannot resolve. On 12,000 real unpatched workflow files from our study, WorkflowBP produces a scanner-checked backport for 67.9%. We further submit 176 issues with WorkflowBP-generated backports to real-world projects, 44 of which have been confirmed or fixed by developers.

0.30

Why Do Pull Requests Go Silent? Uncovering the Barriers to Contribution Completion in Open-Source Code Review

cs.SE ·2026-09-18 ·Md Shamimur Rahman, Farhana Akter, Md Mustakim Billah, Zadia Codabux, Chanchal K. Roy
code/data similarity 0.61
Abstract

Pull requests (PRs) underpin pull-based software development by enabling distributed code review and collaborative contribution in open-source projects. Yet many become inactive before integration and are eventually abandoned or closed, wasting contributor and maintainer effort. Although prior work has examined PR abandonment and review delays, less is known about the contribution types, discussion-level barriers, and post-stalling collaboration patterns associated with inactivity. We investigate which PR types most often stall, why inactivity occurs from authors' and reviewers' perspectives, and how stalling relates to later contributor and reviewer engagement. We analyzed 14,234 stalled PRs and 164,562 review comments from 19 popular GitHub repositories using stale-bot workflows. An LLM-based voting classifier categorized PRs by contribution type, while quantitative analysis was combined with qualitative coding of general and inline review discussions. Feature-enhancement and issue-fixing PRs formed the largest share, together exceeding 77% of classified stalled PRs. General comments linked stalling mainly to communication and coordination breakdowns, including missing interaction, delayed feedback, and unclear follow-up. Inline comments showed that inactivity does not always reflect disengagement: many PRs were blocked by technical or dependency issues, including failing checks, configuration problems, compatibility concerns, and environment mismatches. Only 39.56% of contributors later submitted another PR, and reviewer re-engagement with the same contributors was approximately 21%. PR inactivity is a socio-technical coordination problem involving communication, technical readiness, review ownership, and automation practices. We recommend type-aware triage, clearer review feedback, explicit ownership of next actions, CI blocker management, and cause-aware stale-bot interventions.

0.33

MUSE: An Interactive Meta-Agent for Understanding and Steering LLM-powered Data Science Systems

cs.HC cs.AI ·2026-08-17 ·Wei-Hao Chen, Weixi Tong, Yuan Tian, Chenglong Wang, Tianyi Zhang
top venue similarity 0.59
Abstract · To appear in the 39th Annual ACM Symposium on User Interface Software and Technology (UIST '26), November 2-5, 2026, Detroit, MI, USA

Recent advances in large language models have enabled a new class of agentic data science systems that allow users to complete complex data science workflows through natural language. Although these systems can significantly reduce manual effort, it remains difficult to diagnose their behavior and steer the reasoning process when failures or unexpected outputs occur. We present MUSE, an interactive meta-agent that enhances user understanding and control of agentic data science systems by (1) dynamically restructuring low-level execution traces into multiple semantic levels that support navigation from high-level overviews to low-level implementation details; (2) enabling users to reference specific workflow steps in context to ask grounded questions, provide feedback, and revise problematic steps without manually locating relevant execution history; and (3) supporting mixed-initiative steering by surfacing suspicious steps for inspection, scaffolding the repair process, and translating user repair intent into contextualized instructions for the underlying agent. In a between-subjects study (n = 15), MUSE improved task efficiency and increased users' confidence in understanding and steering agentic data science workflows.

0.55

Orchestrating AI-Assisted Code Remediation: Socio-Technical Bottlenecks in a Large Industrial Repository

cs.SE ·2026-09-24 ·Andreas Bexell, Lo Gullstrand Heander, Emma söderberg
similarity 0.59
Abstract · Submitted

Background: Code degradation in large, long-lived codebases is costly to remediate through manual refactoring and opportunistic clean-ups. LLM-based coding assistants can perform mechanical remediation at scale, but their impact on industrial workflows is underexplored. Objective: We investigate how massive AI-assisted code remediation affects build-on-commit continuous integration (CI), code review, and team coordination in a large industrial repository, and which socio-technical bottlenecks constrain such remediation when source editing becomes cheap through AI assistance. Method: We report on a 15-day exploratory single-case field study in which an experienced developer used a command-line AI coding buddy to remediate widespread issues in a closed-source industrial C++ repository. We triangulate Gerrit metadata with a developer diary and team chat, analyzed through descriptive statistics and qualitative coding. Results: AI-assisted remediation rapidly generated hundreds of commits touching thousands of lines, saturating CI and reviewer attention. Naïve per-file commits overloaded build-on-commit CI; Switching to directory-based batching and capping the number of files per change restored throughput, but still required explicit review solicitation, negotiation of acceptable commit granularity, and iterative follow-up to resolve build and static-analysis failures. Conclusion: When mechanical editing is cheap, CI capacity, review effort, and change orchestration become primary bottlenecks. Sustainable AI-assisted remediation in very large repositories requires deliberate control of commit, review, and CI batch granularity and treating semantic change sets, such as ``fix all instances of warning X'', as first-class units of work that can be sliced differently for developers, reviewers, and CI.

0.16

Cross-Ecosystem Packages As Multilingual: Prevalence, Architecture, and Health

cs.SE cs.DL ·2026-09-14 ·Xiangxi Li, Olivier Nourry, Yoshiki Higo, Raula Gaikovina Kula
similarity 0.58
Abstract

Modern software development increasingly relies on using multiple programming languages. Some software packages are published to multiple package ecosystems such as NPM for JavaScript and PyPI for Python. Little is known about cross-ecosystem packages, especially regarding how they are structured. In this paper, we conduct a large-scale empirical study of over six million packages across six major ecosystems to understand 1) how prevalent cross-ecosystem packages are among all packages, 2) whether there are distinct source code architectural patterns that cross-ecosystem packages use, and 3) whether there are correlations between architectural patterns and project health metrics from GitHub. Results indicate that cross-ecosystem packages constitute a small but important, growing fraction of packages. We identify five distinct architectural patterns. For example, packages that implement code generation from a shared source file or use language bindings are associated with significantly higher community visibility and development activity. Based on our findings, we provide implications for package adopters, maintainers, and researchers. We envision our taxonomy being used for future investigations into several aspects of software development, such as the trade-offs between focusing on one language and translating to other languages using bindings, templating, and wrappers, versus using native code and native functions to support additional languages.

0.16

Scientific Data Skills: Enabling Agent-Ready Scientific Data Services at Scale

cs.AI ·2026-08-20 ·Xiaohan Huang, Qingqing Long, Xiaolei Du, Siyu Pu, Jiawen Xu +6 more
benchmark similarity 0.58
Abstract · 15 pages, 5 figures, 5 tables

Scientific data are increasingly used by AI agents, yet existing dataset representations provide limited support for reliable dataset discovery and interpretation, constraining their effective use in scientific workflows. This limitation arises because agents must search across heterogeneous repositories and reconstruct dataset-specific semantics and operating procedures from documentation designed primarily for human use. To address this limitation, we introduce the Scientific Data Skill (SciDSK), an agent-ready representation that packages dataset-specific knowledge and operational guidance as a reusable agent skill. A SciDSK integrates dataset descriptions, scientific context, file organization, task-specific usage procedures, quality checks, and provenance information while retaining the underlying data in its original repository. We define a structured SciDSK specification and develop a systematic construction pipeline that grounds each SciDSK in authoritative dataset records and associated supporting materials. We further establish the Scientific Data Skill Bank, a unified platform that publishes SciDSK resources across six scientific disciplines and supports package access, persistent identification, and traceability to source datasets. We evaluate SciDSK through a retrieval benchmark for dataset discovery and controlled cases for dataset interpretation. On the query retrieval benchmark, Agent-SciDSK achieves 80.77% Hit@1, exceeding Agent-Raw by 9.62 percentage points. Across controlled interpretation cases, the SciDSK condition satisfies 23 of 24 assessment criteria, compared with 22 under the web-page condition. These results indicate that SciDSK improves how agents locate and understand scientific datasets, providing a stronger foundation for actionable scientific data use.

0.17