CS Paper Radar

GitHub Engagement Signals for CVE Prioritization: The GitHub Popularity Metric (GPM)

Jafar Akhoundali, Kristian Rietveld, Olga Gadyatskaya

cs.CR ·submitted 2026-09-25 08:50 UTC · v1

Abstract

Attackers can compromise multiple systems with a single vulnerability, while defenders need to fix all security weaknesses in their systems. This asymmetry puts defenders at a disadvantage. Security vulnerabilities are found at an alarming rate, and patching vulnerabilities is costly and time-consuming; thus, vulnerability prioritization is a must and a time-critical challenge. Many prioritization metrics, such as the CVSS, EPSS, KEV, and SSVC, are currently used, each with different pros and cons, such as openness, degree of automation, time-criticality, coverage, and need for expert input. In this work, we propose the GitHub Popularity Metric (GPM), a fully open, publicly computable prioritization metric based on the popularity of exploits in GitHub repositories. We use GitHub features such as the number of stars, forks, and related unique users to create a metric that indicates the popularity of CVEs across different time frames, both relative to the current time and historically. We compare the proposed metric with various existing vulnerability prioritization metrics and known exploited vulnerabilities and demonstrate that it provides tangible insights for defenders, identifying unique CVEs and inconsistencies in existing methods. The GPM was integrated into the EPSS version 5. \noindent\textbf{Note.} A demonstration based on this work has been accepted to the demo track of the ACM Conference on Computer and Communications Security (CCS) 2026.

Closest papers

Nearest neighbours by meaning across the whole index.

CAPE - Context-Aware Prioritization Engine for Vulnerability Scoring and Management

cs.CR cs.SE ·2026-10-09 ·Rakesh Podder, Viktoria Koscinski, Indrajit Ray
code/data similarity 0.75
Abstract

Vulnerability patching is critical for safeguarding an enterprise's software ecosystem from cyber attacks. However, patching requires adequate preparation for potential system downtime and for unforeseen effects of the patched software on a live system. Thus, effective risk mitigation requires prioritization strategies that explicitly account for the operational context of the affected software. While vulnerability prioritization based on Common Vulnerability Scoring System (CVSS), Exploit Prediction Scoring System (EPSS) and CISA Known Exploited Vulnerabilities (KEV) catalog, can help, their differing goals and methodologies often lead to inconsistent, context-unaware decisions resulting in mis-allocated remediation efforts. In this work, we present the Context-Aware Prioritization Engine (CAPE), a vulnerability analysis framework that enriches each vulnerability scanner report of vulnerabilities (CVEs) with deployment-specific evidence. This evidence is used to perform vulnerability reachability, centrality, and exploitability analysis, producing a ranked priority score for the effective distribution of remediation effort. We evaluate CAPE on 30 open-source agentic applications comprising 6,138 scanner-reported CVEs. Our evaluation shows that a significant majority of CVEs are not reachable and exploitable which results in significant disparities between CAPE and context-free baselines - the CVSS, EPSS, and KEV. It also shows that CAPE consistently surfaces confirmed-exploitable CVEs at higher priority than all baselines, demonstrating that context-aware analysis provides substantially more accurate remediation guidance than context-free severity scoring alone.

0.33

VEX-Bench: Benchmarking LLM Agents for Assessing Exploitability of Software Supply Chain Vulnerabilities

cs.CR cs.LGcs.SE ·2026-09-07 ·Jiahao Shi, Edward Tsien, Yifeng Di, Hongjiao Zhang, Yuan Tang +8 more
top venuebenchmarkcode/data similarity 0.74
Abstract · Accepted to EMNLP 2026

The software supply chain has become an increasingly exposed attack surface because of its reliance on intricate yet fragile dependencies. Existing defenses such as GitHub Dependabot often raise many false alerts because their coarse-grained matching cannot determine whether a vulnerable dependency is actually exploitable. Security analysts typically spend substantial time assessing vulnerability exploitability case by case. Recent LLM agents have emerged as promising candidates for this task given their advanced capabilities in coding and cybersecurity, yet no existing benchmark evaluates them on it. Prior benchmarks target zero-day settings, where agents detect and exploit previously unknown vulnerabilities. In contrast, software supply chain security focuses on how known vulnerabilities in upstream dependencies affect downstream projects. This requires agents to reason across repositories and determine whether an upstream vulnerability is exploitable in the downstream project. To address this gap, we introduce VEX-Bench, the first benchmark for evaluating LLM agents' ability to assess the exploitability of software supply chain vulnerabilities. It contains 75 real-world cases mined from GitHub and labeled by security experts, covering Python, Java, and Go. We evaluate nine models across three agent harnesses. While GPT-5.5 and Claude Opus 4.6 reach approximately 80% F1 on binary vulnerability-status classification, only GPT-5.5 surpasses 70% macro-F1 on fine-grained justification classification. This gap highlights the challenge of moving beyond binary exploitability assessment to identifying fine-grained exploitability reasons. Code and data: https://github.com/steven1518/vex-bench

0.74

Ties at the Top: A Point-in-Time Comparison of CVSS and EPSS at a Fixed Budget

cs.CR ·2026-11-17 ·Gianluca Martinuz, Paolo Ceravolo
similarity 0.70
Abstract · 15 pages, 4 figures

Organizations must decide which vulnerabilities warrant further analysis and protective action when the volume of published vulnerabilities exceeds their analytical capacity. We present a methodology for comparing vulnerability-prioritization strategies under an explicit resource constraint, represented by a fixed number of CVEs selected from a common eligible population. We apply this methodology to the Common Vulnerability Scoring System (CVSS) and the Exploit Prediction Scoring System (EPSS). For CVEs published in the National Vulnerability Database (NVD) from 17 March to 31 December 2025 and from 1 January to 31 March 2026 (Q1), we reconstruct each CVE's scores and status 30 days after NVD publication, using archived source data. Each cohort has one ranking per strategy at a 5% budget. Positives are CVEs added to the Known Exploited Vulnerabilities (KEV) catalog within 180 days after their decision day. The catalog is maintained by the Cybersecurity and Infrastructure Security Agency (CISA). EPSS captures 16 of 40 positives in 2025, versus 5 for CVSS; alternative selections among tied CVEs cannot raise CVSS capture above 9. In Q1, CVSS captures 4 of 11 and EPSS 3, but tie-breaking can reverse this ordering. The score alone leaves most of each CVSS list undecided: the tie-breaking rule allocates 1,678 of 1,919 CVSS places in 2025 and 589 of 758 in Q1 among CVEs tied at score 9.8. Most chosen CVSS scores come from non-NVD providers; disabling those scores reduces CVSS capture from 5 to 1 in 2025 and from 4 to 0 in Q1. A fixed-budget comparison should therefore report its decision cutoff, eligible population, score provenance, and tie-breaking rule.

0.22

An Exploratory Study of Dependabot Cooldown Adoption in Open-Source GitHub Projects

cs.SE ·2026-09-15 ·Hidetake Tanaka, Rikuto Tsuchida, Kazumasa Shimari, Raula Gaikovina Kula, Kenichi Matsumoto
code/data similarity 0.70
Abstract · 32 pages, 11 tables, 2 figures

Automated dependency updates can rapidly propagate malicious package releases before maintainers and the broader community have enough time to detect them. In July 2025, GitHub made Dependabot cooldown generally available as a defense against software supply chain attacks. However, the effects of its early adoption remain unknown. In this exploratory study, we empirically examine how popular open-source GitHub repositories adopt and configure the feature and investigate their motivations. We find that security concerns motivated 83 of 92 adoption events with known motivations. Security linter warnings triggered 43 of 75 security-only adoptions. Among 251 ecosystems within repositories that retained cooldown, 97.2% set a general delay. Of these, 64.3% used seven days, while use of each update type setting was below 10%. Early adopters therefore favor simple default delays over fine-grained controls. These findings suggest that tools could provide robust defaults reflecting ecosystem support and reserve fine-grained controls for dependencies with clear update priorities.

0.63

Vulnerability Localization Benchmark: Measuring Agentic Security Analysis at Repository Scale

cs.CR cs.AI ·2026-09-14 ·Aman Priyanshu, Supriti Vijay, Kimia Majd, Xuhong He, Fraser Burch +6 more
benchmark similarity 0.70
Abstract · 29 pages, 6 figures, technical report for VLoc-bench

Language-model agents increasingly operate over complete software repositories, yet cybersecurity evaluations primarily measure whether they can detect, reproduce, or repair vulnerabilities rather than whether they can locate the relevant code. We study vulnerability localization: given a weakness class and an unfamiliar repository, identify the implementation files associated with that weakness. We introduce the Vulnerability Localization Benchmark (VLoc Bench), comprising 500 real world vulnerabilities from 290 repositories across six package ecosystems and 147 CWE categories. Each task pairs repository snapshots immediately before and after a security fix. On the vulnerable snapshot, an agent receives only the CWE description and read-only terminal access and must return the affected files; on the patched snapshot, it must determine that the recorded vulnerability is no longer present. We evaluate 27 language models and four static-analysis tools under a common agent interface. Repository-scale vulnerability localization remains difficult: the strongest system achieves 0.229 File F1, and 38.4% of tasks receive no correct localization from any evaluated model. We further find that stronger localization does not imply reliable behavior after remediation: systems that identify vulnerable files effectively can still report unsupported locations on patched repositories. These results establish vulnerability localization as a distinct repository-scale capability and provide a setting for studying both how security agents search for vulnerable code and when they should refrain from reporting it.

0.44

One Strategy Does Not Fit All: Rethinking Data Labeling for Just-in-Time Vulnerability Prediction

cs.SE ·2026-11-03 ·Quan Tran, Duong Nguyen, Thanh Le-Cong, Triet Huynh Minh Le, M. Ali Babar +2 more
similarity 0.70
Abstract · 29 pages

Just-In-Time Vulnerability Prediction (JIT-VP) identifies security risks in code changes at commit time, enabling early feedback before vulnerabilities enter software systems. However, its practical reliability is challenged by the difficulty of constructing accurate labels. Multiple strategies with different trade-offs have been proposed to identify vulnerability-fixing commits (VFCs) and vulnerability-introducing commits (VICs) for dataset construction, leading to substantially different datasets. This makes it difficult to separate model performance from dataset construction. Moreover, real-world quality assurance requires not only classification but also prioritizing high-risk commits and localizing vulnerable causes. This paper presents an extensive empirical study of how labeling strategies affect these downstream tasks. We constructed 28 datasets using four VFC and four VIC labeling strategies across OpenSSL and Linux kernel, then evaluated six representative JIT-VP models for classification, prioritization, and localization. Our results show that labeling effectiveness is strongly task-dependent. Manual VFC labeling combined with vulnerability-specific VIC labeling provides the most reliable performance for classification and prioritization, significantly outperforming 11 of 15 alternatives. Meanwhile, precision-first VFC labeling combined with LLM-assisted VIC labeling performs best for localization, significantly outperforming 7 of 15 alternatives. These findings demonstrate that no single labeling strategy is universally optimal and highlight the importance of task-oriented dataset construction.

0.23

Benchmarking Automated Security Patch Backporting: How Far Are We?

cs.SE cs.AIcs.CR ·2026-08-18 ·Jincheng Yang, Yulong Fu, Chengwei Liu, Lyuye Zhang, Fangyuan Zhang +3 more
top venuebenchmarkdataset similarity 0.69
Abstract · 13 pages, 3 figures. Accepted at ASE 2026. Artifact: https://doi.org/10.5281/zenodo.21785770

Automated security patch backporting is critical for mitigating N-day vulnerabilities. Recent tools report success rates above 80% on their respective datasets. However, these evaluations are often confined to homogeneous environments, such as one repository or specific project versions. Consequently, it remains unclear how well these tools generalize beyond their originally targeted scenarios. We present Porting Benchmark, a curated dataset of 1,234 security patch backporting cases spanning cross-version, cross-branch, and cross-repository scenarios, paired with a common evaluation framework. Using this benchmark, we evaluate five tools spanning program analysis, LLM prompting, and LLM agents under aligned settings. Our results show that aligned evaluation changes the apparent performance landscape: PortGPT and TSBPort remain comparatively strong on the Replication Dataset, while FixMorph and Mystique degrade substantially under the common protocol. Performance degrades sharply on structurally complex patches: the best commit-level success rate falls from 85.2% on Type-I patches to 24.0% on Type-IV. We identify four root-cause categories (missing target API awareness, cross-version semantic mismatch, non-local dependency propagation failure, and patch construction or localization failure) and derive concrete directions for next-generation tool design. On a 45-case dynamically validated subset with verified test cases and constructed POCs, we further observe that reference-based benchmark scores do not fully capture real-world remediation: exact match sharply under-credits harder target adaptations, while executable validation reveals residual integration failures in the target that static reference agreement misses. Executable-feedback refinement provides limited but measurable recovery on the hardest executable cases.

0.62

Raise or Check? Predicting the Necessity of Additional Fuzzing for Continuous Fuzzing

cs.SE cs.CR ·2026-10-20 ·Riku Kato, Tatsuya Shirai, Olivier Nourry, Yutaro Kashiwa, Kenji Fujiwara +2 more
datasetcode/data similarity 0.69
Abstract

Continuous fuzzing is used for vulnerability detection in Open Source Software (OSS). Large-scale services such as Google's OSS-Fuzz fuzz over 1,300 projects and have revealed more than 10,000 vulnerabilities. Despite this, a considerable time lag often exists between when a vulnerability is introduced and when it is detected (i.e., the window of exposure). Given the computational cost of additional fuzzing, we aim to allocate fuzzing resources efficiently using vulnerability-risk estimates. We propose RAISE (Risk-Aware Intensive Supplemental Execution Framework), a Just-In-Time (JIT) framework that prioritizes additional fuzzing for high-risk project-days. RAISE performs binary classification at the project-day granularity to estimate whether a vulnerability is introduced on a given day. We construct a dataset combining vulnerability-introducing commits from the OSV database with daily coverage and change metrics from OSS-Fuzz, covering 15 C/C++ projects and 15,210 project-days, of which 347 (2.28%) are vulnerability-introducing. In a cross-project validation setting, our best model, a Random Forest combining VCCFinder-based static features with dynamic fuzzing features, achieves a ROC-AUC of 0.74, substantially outperforming a random baseline; coverage metrics, led by PatchCoverage, are the most important features. In a simulation over 968 vulnerabilities comparing five allocation strategies, Simple-Regression was significantly more cost-efficient than the prediction-agnostic heuristics, spending 91.60 additional fuzzing sessions per detected vulnerability, roughly half the cost of the Median heuristic (171.89). RAISE shows that JIT vulnerability prediction with coverage-aware fuzzing features can effectively guide risk-aware allocation of continuous fuzzing resources.

0.30