CS Paper Radar

Layered Reproducibility for High-Performance Computing Applications: The Feel++ and Ktirio Urban Buildings Case Study

Javier Cladellas, Vincent Chabannes, Christophe Prud'Homme

cs.DC ·submitted 2026-10-30 15:15 UTC · v1
benchmark

Abstract

The High-Performance Computing (HPC) community struggles with reproducibility due to complex software stacks, heterogeneous hardware, and expensive validation cycles. This contribution presents a layered reproducibility strategy in which Feel++ provides a traceable software supply chain and Ktirio Urban Buildings (KUB) builds on it to manage versioned urban-building data, executable simulations, and checksum-verifiable result artifacts. Feel++ treats continuous integration (CI) plans, CMake presets, packaging manifests, public software metadata, operations commands, containers, software bill of materials (SBOM)/provenance metadata, and Feel++ Benchmarking reports for Feel++ and KUB cases as repositoryowned artifacts. KUB applies this foundation to City Energy Model Database (CEMDB) datasets, manifests, Findable, Accessible, Interoperable, and Reusable (FAIR) packaging, and verifiable outputs for computationally expensive simulations. The case study shifts reproducibility from late manual packaging to an artifact chain linking source commits, packages, containers, benchmark configurations, dataset versions, provenance records, and generated results.

Closest papers

Nearest neighbours by meaning across the whole index.

Reproducibility is Not Enough: Artifact Verifiability in Decentralized-Build Package Ecosystems

cs.SE cs.CR ·2026-08-18 ·Oreofe Solarin, Kelechi Kalu, James C. Davis, Paschal Amusuo
similarity 0.61
Abstract · 12 pages, 6 figures, 3 tables

Reproducible and verifiable builds increase trust in distributed software artifacts by enabling independent parties to detect artifacts produced by compromised build or release pipelines. However, artifact verification requires more than deterministic builds: a verifier must also recover the source state, build environment, dependencies, and build instructions that produced the artifact. Decentralized-build ecosystems make this difficult because artifacts are produced through heterogeneous tools, maintainer-controlled workflows, and fragmented metadata. As a result, it remains unclear how often artifacts in these ecosystems can be independently verified. This paper studies artifact verifiability across four popular decentralized-build package ecosystems. We define an independent verifier model that relies only on registry-derivable metadata and an artifact comparison model with tiered equivalence levels. We implement these models in an Artifact Verification Pipeline and use it to measure artifact verifiability across the target ecosystems. Our results show that, beyond build determinism, verifiability is limited by missing source and build metadata, implicit release transformations, and unconventional build practices. Provenance attestations and embedded VCS metadata improve verification, but they do not provide complete rebuild specifications. These findings identify concrete metadata gaps and ecosystem-level changes needed to make artifact verification practical at package-registry scale.

0.51

ModBench: A Pipeline for Building Modelica Benchmark Datasets Mined from Library Repositories

cs.SE ·2026-08-17 ·Masoud Sadrnezhaad, Martin Sjölund, Adrian Pop, José Antonio Hernández López, Torvald Mårtensson +1 more
top venuebenchmarkcode/data similarity 0.59
Abstract · Extended abstract accepted at SAM 2026, co-located with MODELS 2026. To appear in the ACM/IEEE MODELS 2026 Companion Proceedings

Research on equation-based cyber-physical systems modeling languages, such as Modelica, is constrained by the lack of curated benchmark datasets. This limits empirical insight into the evolution and development of models. We address this gap with ModBench, a pipeline that mines Git repositories of Modelica libraries to produce benchmark datasets of model snapshots. The pipeline (1) filters repository commits to retain human-authored, Modelica-relevant revisions; (2) extracts simulation-eligible classes; and (3) builds canonical representations of Modelica classes. For empirical validation, we applied ModBench to the Modelica Standard Library (MSL) and report the resulting dataset, spanning the full commit history (since Modelica language v3), with 85,562 distinct class snapshots, and links enabling traceability to original models and Git metadata. The dataset, its API, and the data generation pipeline are publicly available to support future research on model evolution analysis, compiler testing, and automated model repair or generation.

0.77

Vibenix: Agentic Software Packaging and Maintenance with Nix

cs.SE ·2026-10-17 ·Martim Monis, Martin Schwaighofer, Nuno Saavedra, Alexandra Mendes, Rene Mayrhofer +1 more
code/data similarity 0.58
Abstract

Implicit dependencies, uncontrolled network access, and other build-environment impurities create intransparent software supply chains. Identifying every dependency affecting a build is a difficult task. Functional package management addresses this through reproducible packaging, ensuring software relies exclusively on declared, hermetically isolated inputs. However, creating build recipes remains complex and time-consuming, posing a significant barrier to adoption for developers and maintainers. We present Vibenix, an open-source AI-powered agent that bridges this gap by automatically generating Nix packaging expressions. Vibenix uses a large language model (LLM) to iteratively refine recipes through a rule-guided build-repair feedback loop. On 466 packaging tasks from our new NixBench dataset, Vibenix produced buildable packages in 95.3% of cases using Claude Haiku 4.5, compared with 5.6% for a state-of-the-art LLM-based single-shot packaging tool. Manual validation of 100 packages revealed that 47% of the buildable packages were initially functionally correct; a post-build refinement process based on the evaluator-optimizer pattern increased functional correctness by 35 percentage points. On our new NixBench-Repair dataset of package maintenance tasks, Vibenix achieved an 87.6% success rate. These results demonstrate how unreliable LLMs can generate rigorously defined, complete, auditable dependency trees for real-world software, reducing maintainer burden while providing a trustworthy foundation for secure software distribution.

0.29

Meta Build Systems as Logic Programs

cs.PL cs.SE ·2026-11-13 ·Alex Reinking
similarity 0.57
Abstract · 60 pages, 4 figures; includes supplementary material

Meta build systems such as CMake, Meson and Bazel derive command graphs from project descriptions, user choices and machine facts. Imperative scripts, search paths and special cases make routine decisions interact poorly: search order can choose compatible packages without an explicit project choice, packagers must patch author flag preferences, and cross-builds need annotations or nested build trees for tools run during the build. Our central insight is to express configuration as relational inference: objects follow from sources, flags and libraries from dependencies, and language standards from transitive requirements. We introduce buildlog, a meta build system whose stratified logical core isolates build derivation from impure environmental probes. It rejects ambiguity in toolchain or package selection and explains derived facts through provenance. Author constraints and user preferences use distinct, statically checked input relations. Libraries can ship consumer rules, including Qt's moc/rcc rules, using the same facts for installed and built code. Using a compiled program in a build rule lets buildlog infer its build-machine configuration. We evaluate buildlog on eleven case studies of ten projects, validating configurations through execution and artifact checks while reporting remaining flag differences and platform limits. In Halide, generators link an LLVM-based compiler and run during the build to emit WebAssembly or Android code; where CMake requires nested builds, buildlog derives build-machine and target-platform configurations in one graph. A supporting formal account establishes termination, within-round probe-order independence, and agreement between fresh and validated cached runs under explicit finiteness and probe obligations. The results show that confining impurity to fact discovery yields a small, uniform and explainable configuration semantics.

0.45

Live Architecture Models for Cloud-Native Architecture-as-Code: Early Results from Kubernetes Conformance Checking

cs.SE ·2026-09-28 ·Denis Mamatin, Andrey Salov
code/data similarity 0.57
Abstract · 16 pages, 5 figures, 9 tables. Code and evaluation artifacts: https://github.com/AsakoKabe/archer-kdl

Infrastructure drift can separate a running Kubernetes system from its documented architectural intent. This paper investigates live architecture models: editable architectural representations connected to selected runtime facts through explicit correspondences and recurring conformance checks. The approach is instantiated through an Archer-specific subset of Kubernetes Deployment Language (KDL) and Archer, a VS Code prototype with synchronized textual and graphical views. Snapshot recovery extracts selected Kubernetes facts into KDL; periodic and on-demand read-only checks report model-cluster inconsistencies without enforcing or repairing deployment state. We assess feasibility on three feature-selected Kubernetes example applications under an author-defined protocol, reporting precision and recall for snapshot recovery and selected inconsistency detection. Recovery scores can be reproduced from saved ground-truth and recovered models; detection uses a documented manual perturbation protocol. The results support feasibility within the evaluated KDL scope. They do not establish comparative superiority, detection of arbitrary production drift, scalability, or developer benefit. Storage coverage and ingress-host representation/default handling remain limitations.

0.27

Joule-Profiler: Profiling the Energy Consumption of Build Automation Tools Made Easy

cs.SE cs.DCcs.PF ·2026-09-25 ·Jérémy Woirhaye, François Gibier, Romain Rouvoy
code/data similarity 0.56
Abstract · 4 pages, 2 figures. Submitted to the ICSE 2027 Tool Demonstration and Data Showcase Track

Build pipelines are integral to modern software development, yet their energy footprint remains largely invisible to practitioners. Existing CI energy tools either rely on model-based estimation (due to hardware access restrictions in cloud runners) or report only total pipeline energy without decomposing it into meaningful phases. Joule-Profiler is an open-source command-line tool for Linux that measures hardware energy consumption via Intel RAPL (CPU), NVML (NVIDIA GPU), and attributes it to user-defined program phases by monitoring standard output. In this tool paper, we demonstrate Joule-Profiler in the context of Maven build pipelines using Google Gson as a case study. By applying a token pattern-matching Maven plugin invocations, we analyze builds across the 5 most recent Gson releases into per-phase energy profiles, comparing cold builds (empty local repository) and warm builds (cached dependencies).

0.30

Collecting Infrastructure-Level Measures for Energy Analysis in Modern Computing Infrastructures

cs.DC ·2026-11-17 ·Isac Pasianotto, Stefano Cozzini, Niccolo Tosato
top venuedataset similarity 0.56
Abstract · Accepted version. Published in SC Workshops '26 (Chicago, IL, USA, 2026). Version of Record: https://doi.ieeecomputersociety.org/10.1109/SCW…

Modern HPC facilities are complex environments in which all components are tightly coupled; consequently, precisely modeling and accounting for energy consumption requires approaches that go beyond instrumenting individual compute nodes. Aiming for a cluster-wide vision, we designed a scalable data collection pipeline that aggregates fine-grained measurements from multiple subsystems-spanning component loads, software and hardware counters, sensors and energy-ensuring consistent sampling frequencies and synchronized data acquisition across the infrastructure. We present a dataset of comprehensive metrics collected at 1 Hz over one month from a production research data center supporting heterogeneous workloads, including HPC, Kubernetes services, and distributed storage. To illustrate what this approach makes possible in practice, we analyze two case studies-distributed I/O and large-language-model inference-showing how cluster-wide measurement reveals system-level dynamics missed by localized instrumentation. The proposed dataset and collection methodology provide a foundation for future work on predictive modeling, energy-aware resource management, and more efficient computing infrastructures.

0.44

OpenRE-Bench: A Benchmark Harness for Multi-Agent Requirements Engineering with Mined GitHub Issues

cs.SE ·2026-11-10 ·Haowei Cheng, Milhan Kim, Bowen Jiang, Weixing Zhang, Yuhong Fu +3 more
benchmarkcode/data similarity 0.56
Abstract

OpenRE-Bench is an open-source benchmarking infrastructure for large language model (LLM) based multi-agent requirements-engineering (RE) frameworks. Such frameworks are increasingly common, but they use different agent roles, negotiation policies, and output shapes, and are typically evaluated on their own runtimes, metrics, and small sets of curated system descriptions, which makes fair, reproducible comparison difficult. OpenRE-Bench addresses this with four reusable pieces: a shared run harness with one input, phase-artifact, and metric contract; an adapter mechanism through which a framework joins by implementing a single adapter, leaving cases, metrics, and other adapters untouched; a reproducible GitHub issue-mining pipeline that grows the corpus from real feature requests; and a versioned case suite. As a demonstration, we mine 3,000 fetched issues across five projects (three languages, five domains) into five cases, package them with five hand-built cases as the compact v1.0 suite, and run three heterogeneous frameworks (MARE and iReDev as paper-faithful reimplementations, plus QUARE) under one protocol. OpenRE-Bench originated as the evaluation back-end of our QUARE study and has since been reused by two further studies of our group. It is released with a pinned environment, a Docker image, frozen API snapshots, a 651-test suite, and the full pool of 1,396 filtered issues behind the mined cases.

0.30