CS Paper Radar

Evaluating the NIST Bugs Framework Against CWE as a Successor for Automated Vulnerability Classification

Md Nazmul Hoque, Shaswata Mitra, Subash Neupane, Sudip Mittal, Shahram Rahimi

cs.CR cs.SE ·submitted 2026-09-14 23:18 UTC · v1
code/data

Abstract

Vulnerability classification based on root cause weaknesses is essential for numerous cybersecurity activities, where the Common Weakness Enumeration (CWE) serves as a public repository of such flaws. However, its overlapping entries create a non-orthogonal structure. The result is the same vulnerability being mapped to multiple weaknesses, complicating Root Cause Analysis (RCA) and triage. To address this, NIST Special Publication 800-231 introduces the Bugs Framework (BF), which organizes vulnerabilities into <cause, operation, consequence> triples and links such triples into a causal chain, so that a vulnerability carries its root cause and its sink together instead of a single terminal label. To date, however, BF has been specified but not evaluated regarding its performance against the challenges to automated classification. The evidence required for adoption has not been investigated empirically. We evaluate BF as a classification target and a complement to CWE using a systematically screened corpus of automated Common Vulnerabilities and Exposures (CVEs) linked to CWE research. We assess the reproducibility of CVE-to-BF classification through two evaluations. The first is qualitative: an anonymized inter-rater study in which 2 subject-matter experts (SMEs) independently mapped 13 CVEs onto the four BF axes. Annotators showed strong agreement on the cause and operation axes, while the attribute axis indicated fair agreement. We also tested our automated framework across two large language model (LLM) deployments under different budgets for reproducibility analysis. Despite limitations, such as evidence availability and the absence of retrievable fix commits for closed-source software, our findings support the claim that BF is a more structured and automation-friendly framework than CWE. Our exploration reveals specific gaps in BF, including under-specified guidance on attributes.

Comment
48 pages, 21 figures, 11 tables, code link: github.com/shaswata09/cve2bf

Closest papers

Nearest neighbours by meaning across the whole index.

How Reliable Are NVD CWE Labels? A Large-Scale Semantic Audit with Seclometry

cs.CR cs.SE ·2026-08-22 ·Yu Nong, Yao Du, Majid Behravan, Haipeng Cai
benchmarkcode/data similarity 0.74
Abstract

CWE labels in the National Vulnerability Database (NVD) are widely treated as ground truth for vulnerability search, scanner evaluation, benchmark construction, learning-based security tools, and vulnerability prioritization. Yet their reliability has not been systematically measured at scale, despite growing concerns about NVD's enrichment backlog and anecdotal reports of inaccurate, ambiguous, or missing labels. This paper presents a large-scale, code-semantics-grounded measurement of CWE labeling quality in NVD. We build CWEAgent, a validated auditing instrument based on seclometry, a structured representation of vulnerability semantics that captures the root cause, trigger condition, violated security property, exploit mechanism, and impact of vulnerable code. On a manually curated benchmark of 100 open-source CVEs, CWEAgent achieves 85% top-1 accuracy and 92% ambiguity-aware accuracy. Applying CWEAgent to 15,556 open-source CVEs disclosed from 2017-2026, we find that only 49.70% of NVD CWE labels exactly match the code-grounded label. Another 31.37% are defensible alternatives under taxonomy ambiguity, while 3.63% are evidence-inconsistent likely errors. Label reliability varies sharply by assigning organization and weakness type, and apparent project- or language-level differences are largely composition effects of those underlying weakness types. Evidence-inconsistent labels have also increased over time. Through manual review of 434 confirmed mislabels, we identify six recurring error patterns, showing that CWE noise is a structural problem in vulnerability metadata rather than isolated annotation mistakes.

0.28

Predicting CVSS and CWE from CVE Descriptions with Fine-Tuned Small Language Models

cs.LG cs.CR ·2026-10-16 ·Tara Dixit, Jessie Dong, Summer Li
similarity 0.70
Abstract · 23 pages, 5 figures, 21 tables

Until April 2026, the National Vulnerability Database labeled nearly every published Common Vulnerabilities and Exposures (CVE) entry with a Common Vulnerability Scoring System (CVSS) v3.1 vector, a severity tier, and a Common Weakness Enumeration (CWE) weakness type. Now, NIST enriches only the high priority CVEs, leaving many newly published vulnerabilities unlabelled with the classifiers that security teams use for patch triage. In CVE enrichment, we predict the eight base components of CVSS v3.1 vector, base score, severity tier, and CWE from the vulnerability's textual description alone. To prevent leakage between near-duplicate descriptions, all systems are trained only on CVEs published in 2022 and 2023. Results are evaluated on all 36,325 usable 2024 CVEs in the pinned NVD snapshot. All scores are either directly predicted, or calculated from predicted vectors using the CVSS formula. A fine-tuned 3.8B-parameter model Phi-3.5-mini, trained with LoRA in 2.5 hours on a single Apple M5 Pro chip, achieves 69.9% accuracy on severity and 70.9% on CWE. It matches the official vector exactly for 48.8% of test CVEs, outperforming TF-IDF logistic regression by 2.8 to 4.7 points on all accuracy metrics based on zero-exclusion paired bootstrap intervals. When we evaluate the base model with a zero-shot prompt, it achieves 20.8% accuracy on severity in a sample of 2,000 CVEs. In this same sample, its stated scores match its own vector precisely for only 3.4% of CVEs. However, deriving scores from predicted vectors boosts the critical recall of TF-IDF logistic regression from 50.7% to 73.3% with no additional cost. And since the errors of the fine-tuned model and TF-IDF affect different CVEs, at least one system matches the correct severity tier for 75.8% of test CVEs.

0.09

CAPE - Context-Aware Prioritization Engine for Vulnerability Scoring and Management

cs.CR cs.SE ·2026-10-09 ·Rakesh Podder, Viktoria Koscinski, Indrajit Ray
code/data similarity 0.70
Abstract

Vulnerability patching is critical for safeguarding an enterprise's software ecosystem from cyber attacks. However, patching requires adequate preparation for potential system downtime and for unforeseen effects of the patched software on a live system. Thus, effective risk mitigation requires prioritization strategies that explicitly account for the operational context of the affected software. While vulnerability prioritization based on Common Vulnerability Scoring System (CVSS), Exploit Prediction Scoring System (EPSS) and CISA Known Exploited Vulnerabilities (KEV) catalog, can help, their differing goals and methodologies often lead to inconsistent, context-unaware decisions resulting in mis-allocated remediation efforts. In this work, we present the Context-Aware Prioritization Engine (CAPE), a vulnerability analysis framework that enriches each vulnerability scanner report of vulnerabilities (CVEs) with deployment-specific evidence. This evidence is used to perform vulnerability reachability, centrality, and exploitability analysis, producing a ranked priority score for the effective distribution of remediation effort. We evaluate CAPE on 30 open-source agentic applications comprising 6,138 scanner-reported CVEs. Our evaluation shows that a significant majority of CVEs are not reachable and exploitable which results in significant disparities between CAPE and context-free baselines - the CVSS, EPSS, and KEV. It also shows that CAPE consistently surfaces confirmed-exploitable CVEs at higher priority than all baselines, demonstrating that context-aware analysis provides substantially more accurate remediation guidance than context-free severity scoring alone.

0.33

An Empirical Study on CVE-based Vulnerabilities Propagation from one System to another System (IntraProject vs InterProject)

cs.CR cs.SE ·2026-11-10 ·Junaid Akram, Ali Hassan, Danish Vasan
code/data similarity 0.68
Abstract · 17 Pages, 14 Figures

Vulnerabilities in applications, including browsers, media players, online services and document readers, etc. are often exploited and cause a serious damage. The vulnerabilities could exist in your system maybe because of some coding errors, design oversights, outdated software, AI generated codes or maybe transferred from another system during development (e.g., code clones). The propagation of software vulnerabilities is a critical concern in cybersecurity. This study examines the root cause problem of vulnerabilities propagation with experiments. We analyzed the propagation of CVE (Common Vulnerabilities and Exposures) based vulnerabilities within and between different software projects (IntraProject vs. InterProject). We performed several experiments on CVE based vulnerability dataset (retrieved from NVD source) by using our vulnerability detection tool (SQVDT). In the results, we discovered similar patterns of vulnerable code in several very famous recently used software, including latest version of Linux, Android operating systems, and in thousands of open source projects available on GitHub. We found several very high severity level (CVSS) vulnerabilities that propagated from one system to another system, just because of copy/paste (reuse) the code. Furthermore, we performed an empirical evaluation and verification of these vulnerable patterns that propagated to other systems by also considering the frequency of their propagation. Finlay, we recommended the impact of these vulnerabilities propagation on the interconnected systems.

0.43

Where AI Vulnerability Discovery Harnesses Go Quiet: Structural False Negatives and Hidden Vulnerabilities

cs.CR ·2026-10-21 ·Gadi Evron
code/data similarity 0.68
Abstract

LLM-based vulnerability discovery harnesses can miss real, reachable vulnerabilities because of how the harness around the model is built. Steps that select code, filter it, and store and merge findings can drop a vulnerability before or after the model runs, independently of the model's capability, and the report usually gives no sign that this happened. These drops come from engineering choices, and errors. They leave vulnerabilities undetected, and an attacker who knows the harness's rules can also use them to hide a vulnerability on purpose. Understanding these drops shows how to build harnesses that fix or disclose them and are harder to evade. Starting from the two harnesses we maintain, Knostic's OpenAnt and raptor, and then examining 10 additional open-source harnesses, we mapped 13 points where such drops occur and organized them into a taxonomy. Using each harness's own code, we show a single file that six harnesses dropped before their model saw its code, five of them without notice. We propose engineering failure patterns mapped to these drops, and relate them to Ptacek and Newsham's work on intrusion detection evasion. The paper also covers content-steered verdicts and prompt injection, and gives design principles and tests for harnesses that report accurately what they analyzed. The reliability findings likely apply to other LLM harnesses, beyond vulnerability research and application security, and the evasion findings apply wherever an attacker can write or influence the code being checked. We also argue that harnesses built to be descriptive can, in effect, be prescriptive. Rules written to describe typical code and typical model replies end up deciding what is analyzed, and anything that does not conform is treated as absent or as "no vulnerability."

0.33

LLMVul: A Vulnerability-Labeled Dataset of LLM-Generated C/C++ Functions from Real Production Repositories

cs.SE ·2026-09-10 ·Mohammad Farhad, Shuvalaxmi Dass
datasetcode/data similarity 0.66
Abstract

Large language models (LLMs) are increasingly used to generate and assist with software development, yet existing vulnerability datasets largely focus on human-written code or controlled prompting environments. This limits the ability to study security weaknesses in LLM-generated code as it appears in real-world software projects. We present LLMVul, a vulnerability-labeled dataset of LLM-generated C/C++ functions mined from real production repositories. We mine AI-assisted development activity from GitHub over a 4 year period, from November 13, 2022 to September 3, 2026, using provenance signals such as commit metadata and AI-related authorship evidence. After filtering and deduplication, LLMVul contains 21,430 unique C/C++ functions from 226 repositories, together with repository, commit, function, provenance, and AI-tool metadata. We establish vulnerability labels using an ensemble of complementary static-analysis and pattern-based techniques and assign Common Weakness Enumeration (CWE) categories to confirmed vulnerable functions. To assess labeling reliability, we additionally conduct independent manual annotation and measure inter-rater agreement using Cohen's kappa ($k=0.79$). LLMVul contains 1,540 ensemble-vulnerable functions spanning 17 unique CWE categories, providing substantially more real-world LLM-generated vulnerable C/C++ functions than existing vulnerability-oriented LLM code benchmarks. By preserving both code-level vulnerability labels and generation/provenance metadata, LLMVul enables reproducible research on vulnerability detection, security evaluation of LLM-generated code, and analysis of vulnerability patterns in AI-assisted software development. The LLMVul dataset is publicly available at https://doi.org/10.5281/zenodo.22668216.

0.29

Ties at the Top: A Point-in-Time Comparison of CVSS and EPSS at a Fixed Budget

cs.CR ·2026-11-17 ·Gianluca Martinuz, Paolo Ceravolo
similarity 0.66
Abstract · 15 pages, 4 figures

Organizations must decide which vulnerabilities warrant further analysis and protective action when the volume of published vulnerabilities exceeds their analytical capacity. We present a methodology for comparing vulnerability-prioritization strategies under an explicit resource constraint, represented by a fixed number of CVEs selected from a common eligible population. We apply this methodology to the Common Vulnerability Scoring System (CVSS) and the Exploit Prediction Scoring System (EPSS). For CVEs published in the National Vulnerability Database (NVD) from 17 March to 31 December 2025 and from 1 January to 31 March 2026 (Q1), we reconstruct each CVE's scores and status 30 days after NVD publication, using archived source data. Each cohort has one ranking per strategy at a 5% budget. Positives are CVEs added to the Known Exploited Vulnerabilities (KEV) catalog within 180 days after their decision day. The catalog is maintained by the Cybersecurity and Infrastructure Security Agency (CISA). EPSS captures 16 of 40 positives in 2025, versus 5 for CVSS; alternative selections among tied CVEs cannot raise CVSS capture above 9. In Q1, CVSS captures 4 of 11 and EPSS 3, but tie-breaking can reverse this ordering. The score alone leaves most of each CVSS list undecided: the tie-breaking rule allocates 1,678 of 1,919 CVSS places in 2025 and 589 of 758 in Q1 among CVEs tied at score 9.8. Most chosen CVSS scores come from non-NVD providers; disabling those scores reduces CVSS capture from 5 to 1 in 2025 and from 4 to 0 in Q1. A fixed-budget comparison should therefore report its decision cutoff, eligible population, score provenance, and tie-breaking rule.

0.15

Investigating Developer-Reported Software Security Testing Challenges

cs.SE cs.CR ·2026-09-10 ·Md Erfan, Ahmed Ryan, Md Rayhanur Rahman
similarity 0.66
Abstract · 34 pages

Software security testing (SST) is essential for identifying vulnerabilities and improving software security, but developers often face practical challenges when selecting tools, configuring test environments, interpreting scanner outputs, testing authentication workflows, and acting on reported vulnerabilities. This study empirically characterizes developer-reported SST challenges in Stack Overflow (SO) discussions and examines their prevalence, difficulty, temporal evolution, co-occurrence, and taxonomy stability. We analyze 17,743 SO questions collected using SST-related keywords. Through manual labeling, we identify 582 SST-related questions and construct a taxonomy of 8 categories and 31 subcategories. We then analyze these challenges using prevalence, difficulty, correlation, temporal trend, co-occurrence, and held-out stability analyses. The results show that developers frequently discuss security finding interpretation and reliability, security testing guidance and tool selection, authentication and authorization testing, and security tool integration and automation. Validation-related challenges often require more technical context and longer resolution time. Security finding interpretation and remediation-related actionability show increasing trends over time. Co-occurrence analysis shows that false positives frequently appear with explainability, while integration challenges often appear with tool suggestions and documentation/resources. Overall, developer-reported SST challenges extend beyond vulnerability detection and include workflow, configuration, interpretation, and remediation concerns. These findings can help researchers, practitioners, educators, and tool providers improve SST usability, documentation, result interpretation, and remediation support.

0.25