CS Paper Radar

Predicting CVSS and CWE from CVE Descriptions with Fine-Tuned Small Language Models

Tara Dixit, Jessie Dong, Summer Li

cs.LG cs.CR ·submitted 2026-10-16 22:46 UTC · v1

Abstract

Until April 2026, the National Vulnerability Database labeled nearly every published Common Vulnerabilities and Exposures (CVE) entry with a Common Vulnerability Scoring System (CVSS) v3.1 vector, a severity tier, and a Common Weakness Enumeration (CWE) weakness type. Now, NIST enriches only the high priority CVEs, leaving many newly published vulnerabilities unlabelled with the classifiers that security teams use for patch triage. In CVE enrichment, we predict the eight base components of CVSS v3.1 vector, base score, severity tier, and CWE from the vulnerability's textual description alone. To prevent leakage between near-duplicate descriptions, all systems are trained only on CVEs published in 2022 and 2023. Results are evaluated on all 36,325 usable 2024 CVEs in the pinned NVD snapshot. All scores are either directly predicted, or calculated from predicted vectors using the CVSS formula. A fine-tuned 3.8B-parameter model Phi-3.5-mini, trained with LoRA in 2.5 hours on a single Apple M5 Pro chip, achieves 69.9% accuracy on severity and 70.9% on CWE. It matches the official vector exactly for 48.8% of test CVEs, outperforming TF-IDF logistic regression by 2.8 to 4.7 points on all accuracy metrics based on zero-exclusion paired bootstrap intervals. When we evaluate the base model with a zero-shot prompt, it achieves 20.8% accuracy on severity in a sample of 2,000 CVEs. In this same sample, its stated scores match its own vector precisely for only 3.4% of CVEs. However, deriving scores from predicted vectors boosts the critical recall of TF-IDF logistic regression from 50.7% to 73.3% with no additional cost. And since the errors of the fine-tuned model and TF-IDF affect different CVEs, at least one system matches the correct severity tier for 75.8% of test CVEs.

Comment
23 pages, 5 figures, 21 tables

Closest papers

Nearest neighbours by meaning across the whole index.

Towards Early Warning of Real-World Vulnerabilities: An Evidence-Aware Decoupled Method for CVSS Vector Prediction Based on Unstructured Disclosures

cs.CR ·2026-11-04 ·Jianhong Ma, Wenhao Li, Xixing Li, Chenyu Yan, Haowen Chen
benchmarkcode/data similarity 0.76
Abstract · 21 pages, 12 figures; includes 3 pages of supplementary material

Early assessment of Common Vulnerability Scoring System (CVSS) vectors from Open Source Intelligence (OSINT) can reduce the delay between initial public disclosure and official National Vulnerability Database (NVD) scoring. Existing predictors, trained mainly on standardized late-stage descriptions, degrade on noisy and incomplete pre-NVD reports. We introduce PARSE, a vulnerability-disclosure lifecycle model, and VPre, an early assessment framework built on two-level Evidence-Aware Decoupling. Decoupling Level I, Security-Evidence Screening, separates security-relevant evidence from noisy early reports at the report and sentence levels. Decoupling Level II, Evidence-Matched Routing, decouples the prediction task of the eight CVSS metrics into a scattered-evidence class and an explicit-evidence class based on validation-set measurements, matching each with a targeted predictor: large-language-model reasoning for the former, lightweight supervised classification for the latter. On the 851-report held-out test split of the 5,581-report VPre-Bench, VPre achieves a mean absolute error of 0.7239 -- 44.7% below the reproduced BERT-Small-NVD early-transfer baseline; paired timestamps across the full benchmark show an average 38-day lead before official scoring (observed range 21-60 days). Under the 38:1 PARSE-Triage noise setting, STDU's report-level screening reaches 0.98 precision, 1.00 recall, and 0.99 F1. Controlled ablations support the contributions of both decoupling levels.

0.28

How Reliable Are NVD CWE Labels? A Large-Scale Semantic Audit with Seclometry

cs.CR cs.SE ·2026-08-22 ·Yu Nong, Yao Du, Majid Behravan, Haipeng Cai
benchmarkcode/data similarity 0.74
Abstract

CWE labels in the National Vulnerability Database (NVD) are widely treated as ground truth for vulnerability search, scanner evaluation, benchmark construction, learning-based security tools, and vulnerability prioritization. Yet their reliability has not been systematically measured at scale, despite growing concerns about NVD's enrichment backlog and anecdotal reports of inaccurate, ambiguous, or missing labels. This paper presents a large-scale, code-semantics-grounded measurement of CWE labeling quality in NVD. We build CWEAgent, a validated auditing instrument based on seclometry, a structured representation of vulnerability semantics that captures the root cause, trigger condition, violated security property, exploit mechanism, and impact of vulnerable code. On a manually curated benchmark of 100 open-source CVEs, CWEAgent achieves 85% top-1 accuracy and 92% ambiguity-aware accuracy. Applying CWEAgent to 15,556 open-source CVEs disclosed from 2017-2026, we find that only 49.70% of NVD CWE labels exactly match the code-grounded label. Another 31.37% are defensible alternatives under taxonomy ambiguity, while 3.63% are evidence-inconsistent likely errors. Label reliability varies sharply by assigning organization and weakness type, and apparent project- or language-level differences are largely composition effects of those underlying weakness types. Evidence-inconsistent labels have also increased over time. Through manual review of 434 confirmed mislabels, we identify six recurring error patterns, showing that CWE noise is a structural problem in vulnerability metadata rather than isolated annotation mistakes.

0.28

ThreatLens: Evidence-Guided Ranking of High-Priority CVEs

cs.CR ·2026-08-23 ·Soroush Motamedi Sedeh, Panteha Shahrivar, Malaika Qureshi, Ali Devjiani, Mohammad A. Tayebi
top venue similarity 0.73
Abstract · 8 pages, 3 figures, 4 tables. Accepted to the CIKM 2026 Applied Research Track

Security teams must prioritize vulnerabilities before exploitation evidence is complete. Existing signals, such as CVSS, EPSS, advisories, and public exploits, are useful but fragmented and time-sensitive; retrospective rankings can therefore overstate performance by using evidence unavailable at decision time. We present ThreatLens, a simple yet effective and deployment-realistic framework for CVE prioritization. ThreatLens ranks vulnerabilities at each review point using only cutoff-valid evidence and learns from future CISA KEV entries as weak supervision for exploitation relevance. Under forward-in-time, CVE-disjoint evaluation, ThreatLens significantly outperforms CVSS, EPSS, and rule-based evidence-fusion baselines. On the held-out test split, ThreatLens surfaces 80.0% of future KEV CVEs in the top 20, over three times EPSS at the same budget, and reaches 95.9% in the top 50. Early-warning analysis further shows that ThreatLens identifies a substantial fraction of subsequent KEV entries before formal catalog inclusion, supporting timely, evidence-grounded triage.

0.43

Narratives Mislead: An Empirical Study of How LLMs Misjudge CVSS Severity in Bug Bounty Reports

cs.CR cs.SE ·2026-10-17 ·Ruitao Feng, Yuekang Li, Yao Zhang
benchmarktheory similarity 0.72
Abstract · 10 pages,6 figures

Bug bounty platforms receive vulnerability reports faster than vendors can triage them, and Large Language Models (LLMs) are increasingly considered for assigning the Common Vulnerability Scoring System (CVSS) severity that drives prioritization. Prior evaluations score short CVE descriptions and report accuracy, but deployment depends on how models err, not only how often. CVSS gives each metric a formal meaning that departs from how researchers describe impact in narrative reports, and model rationales suggest that errors concentrate where the two diverge; we ask whether this pattern is systematic. From 4,281 publicly disclosed reports mined from Huntr and HackerOne, we build a benchmark of 240 Huntr reports with vendor-validated CVSS vectors and evaluate eight LLMs under four prompting strategies and fine-tuning, measuring errors by severity class and score and analysing sub-vectors on individual reports. The best prompting configuration reaches only 49.58% level accuracy, ten points above always predicting the majority level, and fine-tuning a 7B model raises this to at most 56.25%, an optimistic upper bound. Errors are directional: models overestimate far more often than they underestimate, and they almost never recognize low-severity vulnerabilities. We trace these errors to three semantic boundaries that the specification draws and narrative reports blur: potential versus realized impact, impact magnitude versus authorization scope, and uncertainty versus the worst case. Because the violations originate partly in the report, we close with an exploratory case study: on four reports, a boundary-aligned, exploit-free report structure lowered a frontier model's score where the narrative invited an unclaimed account takeover, but left the metrics a report does not settle open, and the two published labels disagreed on three of the four reports.

0.24

CoSA: Context-Aware Severity Assessment via Context Analysis with Large Language Models

cs.CR cs.SE ·2026-08-14 ·Jinfeng Jiang, Yikun Li, Chengran Yang, Ting Zhang, Wen Bin Leow +4 more
similarity 0.71
Abstract

Accurate vulnerability severity assessment is essential for prioritizing remediation, yet manually assessing Common Vulnerability Scoring System (CVSS) base metrics remains labor-intensive. Existing automated approaches often fail to capture the repository-level evidence required for assessing many CVSS base metrics. Such repository-aware assessment is challenging because relevant evidence is scattered across the entire repository under heavy noise. To address these challenges, we present CoSA, a Context-aware vulnerability Severity Assessment approach that infers CVSS base metrics from repository artifacts. CoSA constructs a code property graph (CPG) and applies a two-stage repository-pruning strategy: lightweight static pruning to preserve structurally proximal context, followed by an agentic large language model (LLM)-guided pruning step to retain CVSS-relevant context while collecting supporting evidence. The LLM then consolidates the retrieved repository context into compact, CVSS metric-wise textual summaries, which are fed into a lightweight transformer predictor. We also construct a higher-quality repository-level dataset comprising 6,816 CVSS labeled instances spanning 90 Common Weakness Enumeration (CWE) types. Experiments on real-world vulnerabilities show that CoSA consistently outperforms function-level and pure-LLM baselines. It improves prediction accuracy by 14.4% and Macro-F1 by 15.3% over the best-performing baseline, suggesting that explicit, metric-oriented repository context retrieval is crucial for practical and reliable automated severity assessment.

0.04

One Strategy Does Not Fit All: Rethinking Data Labeling for Just-in-Time Vulnerability Prediction

cs.SE ·2026-11-03 ·Quan Tran, Duong Nguyen, Thanh Le-Cong, Triet Huynh Minh Le, M. Ali Babar +2 more
similarity 0.71
Abstract · 29 pages

Just-In-Time Vulnerability Prediction (JIT-VP) identifies security risks in code changes at commit time, enabling early feedback before vulnerabilities enter software systems. However, its practical reliability is challenged by the difficulty of constructing accurate labels. Multiple strategies with different trade-offs have been proposed to identify vulnerability-fixing commits (VFCs) and vulnerability-introducing commits (VICs) for dataset construction, leading to substantially different datasets. This makes it difficult to separate model performance from dataset construction. Moreover, real-world quality assurance requires not only classification but also prioritizing high-risk commits and localizing vulnerable causes. This paper presents an extensive empirical study of how labeling strategies affect these downstream tasks. We constructed 28 datasets using four VFC and four VIC labeling strategies across OpenSSL and Linux kernel, then evaluated six representative JIT-VP models for classification, prioritization, and localization. Our results show that labeling effectiveness is strongly task-dependent. Manual VFC labeling combined with vulnerability-specific VIC labeling provides the most reliable performance for classification and prioritization, significantly outperforming 11 of 15 alternatives. Meanwhile, precision-first VFC labeling combined with LLM-assisted VIC labeling performs best for localization, significantly outperforming 7 of 15 alternatives. These findings demonstrate that no single labeling strategy is universally optimal and highlight the importance of task-oriented dataset construction.

0.23

On Predicting Vulnerability Severity Using In-Context Learning: An Industrial Case Study

cs.CR cs.AI ·2026-08-22 ·Daniel Rodriguez-Cardenas, David Nader Palacio, Anna Schmedding, Yiyang Lu, Aadil Mallick +6 more
publishedcode/data similarity 0.71
Abstract · 11 pages, 6 figures

Modern software systems require earlier and more scalable vulnerability severity assessment to reduce exposure to high-impact security flaws. Security analysts typically assign CVSS scores, but this manual triage does not scale with the growth of disclosed vulnerabilities and often depends on cloud LLM services that raise confidentiality concerns. This paper presents an industrial case study on predicting CVSS v3.1 scores directly from vulnerable C/C++ snippets using in-context learning with locally deployable, open-source LLMs. We compare proprietary data with the Big-Vul dataset, showing sufficiently aligned CVSS distributions to justify Big-Vul as a proxy for industrial data when constructing prompt-based testbeds. We then vary in-context configurations and model parameters, evaluating CodeLlama2-7B, CodeLlama2-13B, Mistral-7B, gpt-oss, and GPT4o-mini using mean squared error (MSE) and feasibility metrics. Our results show that medium-sized open-source code models, particularly CodeLlama2-7B, can approximate the best cloud performance for CVSS regression when guided by lightweight, output-constraining prompts, offering a practical, privacy-preserving building block for severity triage in industrial settings.

0.40

Evaluating the NIST Bugs Framework Against CWE as a Successor for Automated Vulnerability Classification

cs.CR cs.SE ·2026-09-14 ·Md Nazmul Hoque, Shaswata Mitra, Subash Neupane, Sudip Mittal, Shahram Rahimi
code/data similarity 0.70
Abstract · 48 pages, 21 figures, 11 tables, code link: github.com/shaswata09/cve2bf

Vulnerability classification based on root cause weaknesses is essential for numerous cybersecurity activities, where the Common Weakness Enumeration (CWE) serves as a public repository of such flaws. However, its overlapping entries create a non-orthogonal structure. The result is the same vulnerability being mapped to multiple weaknesses, complicating Root Cause Analysis (RCA) and triage. To address this, NIST Special Publication 800-231 introduces the Bugs Framework (BF), which organizes vulnerabilities into <cause, operation, consequence> triples and links such triples into a causal chain, so that a vulnerability carries its root cause and its sink together instead of a single terminal label. To date, however, BF has been specified but not evaluated regarding its performance against the challenges to automated classification. The evidence required for adoption has not been investigated empirically. We evaluate BF as a classification target and a complement to CWE using a systematically screened corpus of automated Common Vulnerabilities and Exposures (CVEs) linked to CWE research. We assess the reproducibility of CVE-to-BF classification through two evaluations. The first is qualitative: an anonymized inter-rater study in which 2 subject-matter experts (SMEs) independently mapped 13 CVEs onto the four BF axes. Annotators showed strong agreement on the cause and operation axes, while the attribute axis indicated fair agreement. We also tested our automated framework across two large language model (LLM) deployments under different budgets for reproducibility analysis. Despite limitations, such as evidence availability and the absence of retrievable fix commits for closed-source software, our findings support the claim that BF is a more structured and automation-friendly framework than CWE. Our exploration reveals specific gaps in BF, including under-specified guidance on attributes.

0.23