CS Paper Radar

DCI: Dependency Confidence Index for Assessing Open-Source Dependency Trustworthiness

Clemens Albrecht, Stefan Reitmann

cs.SE cs.CR ·submitted 2026-08-17 11:31 UTC · v1
surveycode/datahot direction

Abstract

Selecting trustworthy open source software dependencies remains a major challenge in software supply chain security. We present the Dependency Confidence Index (DCI), a composite formative index that combines nine empirically weighted trust factors into a single normalized composite score for dependency selection. DCI's trust factors combine insights from a systematic literature review and an exploratory Analytic Hierarchy Process (AHP) survey of ten software developers, highlighting security, source code quality, and project health as the most influential dimensions. Following Goal-Question-Metric methodology, we implemented 12 automated measurements using SonarQube, GitHub APIs, and OpenSSF Scorecard data, deployed in a containerized evaluation platform. We conducted a pilot evaluation of the normalized DCI on 92 popular PyPI packages, observing moderate agreement with OpenSSF Scorecard scores and perfect test--retest reliability. Analysis reveals process-based factors (dependency management, CI) dominate scores on high-quality packages, while security metrics saturate---suggesting DCI's complementary role to existing tools. Our publicly available implementation provides a foundation for open source software trustworthiness research and practical dependency auditing.

Closest papers

Nearest neighbours by meaning across the whole index.

What Is Under the Hood? A Cross-Ecosystem Study of Vulnerable Dependencies in Construction Software

cs.SE ·2026-10-19 ·Mahzabin Tamanna, S M Jamil Uddin
code/data similarity 0.70
Abstract

The construction industry increasingly relies on software built from third-party components, yet the dependency composition and known-vulnerability exposure of construction-related software remain poorly analyzed. In this study, we conduct an empirical, cross-ecosystem investigation of 904 construction-related GitHub repositories spanning six construction functions and a multifunction category. Using commit-pinned repository snapshots and OSV-Scanner, we examine dependency assessability, known-vulnerability prevalence, severity, weakness classifications, and shared upstream exposure. Our results reveal that only 494 repositories (54.6%) completed vulnerability assessment, revealing substantial gaps in dependency visibility. Among successfully scanned repositories, 395 (80.0%) contained at least one dependency version associated with a published vulnerability advisory, and 97.2% of affected repositories contained a High or Critical severity risk. Additionally, Common Weakness Enumeration (CWE) mapping identified recurring resource-consumption, path-traversal, information-exposure, and code-injection weaknesses. Vulnerability burden was uneven, with a median of 50 distinct advisory clusters per affected repository, while just two affected upstream package families collectively reached ~70% of affected repositories. These findings establish an empirical baseline for dependency assessability and known-vulnerability exposure in construction-related open-source software. Our findings underscore the importance of documenting assessment coverage, interpreting advisory matches in their deployment context, and examining shared upstream dependencies when evaluating construction software supply-chain security.

0.20

AgOSS: A Dataset and Multi-Layer Characterization of Open-Source Agricultural Software

cs.SE ·2026-09-02 ·Vatsal Dudhaiya, Mikhail Golovenchits, Aryan Banerjee, James C. Davis
datasetcode/data similarity 0.70
Abstract

Much of agriculture depends on open-source software spanning farm management platforms, cloud services, edge gateways, embedded systems, and field-deployed sensors, forming a domain-specific software supply chain that has drawn little empirical security attention. It is unknown whether this ecosystem's supply chain security posture differs from that of comparable non-agricultural software, and if it does, whether the difference reflects the agricultural domain or the size and maturity of the projects within it. As a step towards securing agricultural open-source software, we present AgOSS, a dataset of 66 repositories across six architectural categories. We assess supply chain security within the dataset via OpenSSF Scorecard, governance metrics, SBOM-based dependency analysis, and KEV matching, and compare against matched non-agricultural controls. We report two findings. First, governance is largely independent of inherited dependency risk. Scorecard tracks community activity, but we detect no association with vulnerability count, density, or known-exploited count, and in regression the exposure signal loads on architectural category rather than domain. Second, agricultural projects score far lower on raw Scorecard, but size and maturity confound the gap: the residual loses significance under matching and regression. Securing this ecosystem means investing in contributor capacity and dependency hygiene, not agriculture-specific controls.

0.20

An Exploratory Study of Dependabot Cooldown Adoption in Open-Source GitHub Projects

cs.SE ·2026-09-15 ·Hidetake Tanaka, Rikuto Tsuchida, Kazumasa Shimari, Raula Gaikovina Kula, Kenichi Matsumoto
code/data similarity 0.67
Abstract · 32 pages, 11 tables, 2 figures

Automated dependency updates can rapidly propagate malicious package releases before maintainers and the broader community have enough time to detect them. In July 2025, GitHub made Dependabot cooldown generally available as a defense against software supply chain attacks. However, the effects of its early adoption remain unknown. In this exploratory study, we empirically examine how popular open-source GitHub repositories adopt and configure the feature and investigate their motivations. We find that security concerns motivated 83 of 92 adoption events with known motivations. Security linter warnings triggered 43 of 75 security-only adoptions. Among 251 ecosystems within repositories that retained cooldown, 97.2% set a general delay. Of these, 64.3% used seven days, while use of each update type setting was below 10%. Early adopters therefore favor simple default delays over fine-grained controls. These findings suggest that tools could provide robust defaults reflecting ecosystem support and reserve fine-grained controls for dependencies with clear update priorities.

0.64

Measuring the Security of the Evolving Software Supply Chain: a Research Agenda

cs.CR cs.SE ·2026-09-08 ·Sarah Meriem Ourari
top venuesurveyposition similarity 0.67
Abstract · Accepted at IEEE ICSME 2026 , 6 pages , 3 figures

Software supply chain security has become increasingly critical due to the widespread reliance on third-party dependencies and the growing attack surface of modern software ecosystems. However, existing quantitative, measurement-based analysis and vulnerability management approaches remain largely fragmented and ecosystem-specific, limiting their ability to provide comparable risk assessments across environments. This paper presents a structured research plan, starting with a Systematization of Knowledge (SoK) to synthesize the current state of research and identify key gaps, highlighting the limitations in dependency modeling and vulnerability propagation analysis, particularly in the treatment of transitive dependencies and their real-world exploitability. Based on these insights, we argue for a unified measurement perspective capable of consistently representing and analyzing the cross-ecosystem dependency structure. We further identify emerging challenges introduced by AI-assisted software development, where coding LLMs are likely to contribute to new dependency patterns that are not captured by traditional Software Composition Analysis (SCA) tools. These shifts motivate a rethink of dependency modeling to account for evolving software-generation practices and their long-term structural impact on software security.

0.34

PatchRisk: Forecasting Future Vulnerability Exposure in Open-Source Dependency Networks

cs.CR cs.SE ·2026-09-12 ·Ashfaq Ali Shafin, Khandaker Mamun Ahmed
benchmarkcode/data similarity 0.67
Abstract · 8 pages

Open-source software ecosystems are web-scale dependency networks. A downstream package can become exposed to security risk not because its own source code changes, but because one of its transitive dependencies later receives a vulnerability advisory. Existing vulnerability-detection work often focuses on whether code is currently vulnerable or whether a known vulnerable dependency is already present. We study a different problem: future transitive vulnerability exposure. Given a package-version dependency graph observed at release time, the task is to predict whether any non-root dependency will receive a vulnerability advisory within a future horizon. This problem is important for Web intelligence and software supply-chain security because it supports proactive dependency triage before future exposure is visible. It is also easy to evaluate incorrectly: current vulnerable dependencies can leak the label, and different versions of the same root package can create package-level memorization across train and test splits. We therefore construct PatchRisk, a leakage-aware benchmark from Open Source Vulnerability advisories and deps.dev dependency graphs for npm and PyPI. The benchmark uses filtration-aware labels, package-disjoint evaluation, temporal testing, and nested 1K, 3K, 5K, and 10K sampling scales. The largest cleaned setting contains 9,007 root package-version graphs spanning 4,157 root packages. We evaluate three feature families: TimeOnly, GraphStruct, and HistoryGraph. On the 10K temporal-test benchmark, HistoryGraph improves AUPRC over the strongest TimeOnly baseline from 0.351 to 0.640 for 90-day forecasting and from 0.473 to 0.813 for 365-day forecasting. The improvement remains stable across smaller scales and package-group shuffle robustness tests.

0.32

Assessing the Construct Validity of Object-Oriented, Class-Level Code Quality Metrics

cs.SE ·2026-09-17 ·Hera Arif, Miikka Kuutila, Paul Ralph
top venuecode/data similarity 0.63
Abstract · Accepted to the ESEIW 2026 ESEM - Technical Track

Background: Code quality metrics are intended to measure latent properties of software source code. Although numerous code metrics have been proposed and used, their construct validity is rarely evaluated. Thus, the extent to which code metrics actually measure what they claim to measure is often unclear. Aim: Drawing from modern measurement theory, we investigate the construct validity of common class-level, object-oriented code quality metrics by identifying their factor structure using Exploratory Factor Analysis (EFA). The metrics were extracted from the Apache Maven project by three software tools: Designite, JHawk, and Understand. The factor structure was later verified using Confirmatory Factor Analysis (CFA) on 22 randomly selected open source projects meeting a predetermined eligibility criteria. Results: 24 code quality metrics that correspond to six constructs: Cohesion, In-Coupling, Out-Coupling, Size, Sub-Inheritance (related to subclasses), and Sup-Inheritance (related to superclasses) were revealed in the underlying factor structure. Ten metrics did not correspond to any known dimension of software quality and were removed in the EFA. Ten additional metrics exhibited low loadings in the CFA, suggesting their removal from the final measurement model. Size, Cohesion, Inheritance, and Coupling were the constructs retained, with subcategories identified for Inheritance and Coupling. Conclusions: Our results strongly support the construct validity of 24 code quality metrics. Coupling and Inheritance are revealed as multidimensional constructs, since they require measuring two different concepts, revealed as sub-categories in our analysis, and Complexity may be better explored in a multilevel model. Overall, our study demonstrates the value of applying modern measurement theory and latent variable modeling in validating software code quality metrics.

0.71

How Reliable Are NVD CWE Labels? A Large-Scale Semantic Audit with Seclometry

cs.CR cs.SE ·2026-08-22 ·Yu Nong, Yao Du, Majid Behravan, Haipeng Cai
benchmarkcode/data similarity 0.63
Abstract

CWE labels in the National Vulnerability Database (NVD) are widely treated as ground truth for vulnerability search, scanner evaluation, benchmark construction, learning-based security tools, and vulnerability prioritization. Yet their reliability has not been systematically measured at scale, despite growing concerns about NVD's enrichment backlog and anecdotal reports of inaccurate, ambiguous, or missing labels. This paper presents a large-scale, code-semantics-grounded measurement of CWE labeling quality in NVD. We build CWEAgent, a validated auditing instrument based on seclometry, a structured representation of vulnerability semantics that captures the root cause, trigger condition, violated security property, exploit mechanism, and impact of vulnerable code. On a manually curated benchmark of 100 open-source CVEs, CWEAgent achieves 85% top-1 accuracy and 92% ambiguity-aware accuracy. Applying CWEAgent to 15,556 open-source CVEs disclosed from 2017-2026, we find that only 49.70% of NVD CWE labels exactly match the code-grounded label. Another 31.37% are defensible alternatives under taxonomy ambiguity, while 3.63% are evidence-inconsistent likely errors. Label reliability varies sharply by assigning organization and weakness type, and apparent project- or language-level differences are largely composition effects of those underlying weakness types. Evidence-inconsistent labels have also increased over time. Through manual review of 434 confirmed mislabels, we identify six recurring error patterns, showing that CWE noise is a structural problem in vulnerability metadata rather than isolated annotation mistakes.

0.28

The Rising Cost of Trust: Practitioners' Trust Signals, Controls, and Responses in the Software Supply Chain

cs.CR ·2026-08-21 ·Ranindya Paramitha, Siri Paidipalli, Laurie Williams, Christian Kästner
code/data similarity 0.63
Abstract

The software supply chain is becoming more complex, and AI is reshaping its threat landscape, e.g., raising concerns about the quality of AI-generated dependencies. Seen through the lens of trust, the stakes of eroding trust in the software supply chain are high, yet we lack an empirical baseline on practitioners' trust. The goal of this study is to aid software practitioners in taking informed actions as trust in the software supply chain evolves, through an interview study with 38 practitioners. We conducted semi-structured interviews with industry and open-source practitioners, focusing on their revealed preferences (the controls they adopted) rather than their stated attitudes, and analyzed the data using thematic analysis grounded in established trust concepts from the social sciences. We find that trust is eroding, which is becoming costly: aware practitioners are accumulating controls. To cope with the rising cost of trust, practitioners automate verification, delegate trust decisions to guardians, or consider exiting the software supply chain entirely. Understanding software supply chain dynamics through the lens of trust provides the vocabulary and concepts (e.g., guardians of trust, system trust, signals) to shape future interventions for a well-functioning supply chain with appropriate levels of trust.

0.41