CS Paper Radar

PatchCon: Patch-aware Weighted Supervised Contrastive Learning for Vulnerability Detection

Chaomeng Lu, Pooya Rostami Mazrae, Bert Lagaisse

cs.SE ·submitted 2026-11-04 14:39 UTC · v1

Abstract

Automated vulnerability detection is critical for securing modern software systems, yet existing learning-based approaches often treat functions as independent samples and underuse the supervision provided by vulnerability-fixing patches. In this paper, we propose PatchCon, a CodeBERT-based patch-aware contrastive learning model for function-level vulnerability detection. PatchCon jointly optimizes vulnerability classification and supervised contrastive learning, and treats vulnerable-patched function pairs as hard negatives with larger contrastive weights to capture subtle security-relevant differences introduced by fixes. To evaluate patch-level discrimination, we use two pairwise metrics that measure whether a model can correctly distinguish vulnerable functions from their patched counterparts. We evaluate PatchCon on PairVul-Linux through extensive experiments, including state-of-the-art comparisons, ablation studies, hyperparameter sensitivity analysis, and statistical significance testing. We also conduct fine-grained analyses by Common Weakness Enumeration (CWE) type, function length, and patch size, and further evaluate PatchCon on Linux kernel vulnerabilities disclosed after the training data cutoff. On PairVul-Linux, PatchCon outperforms representative pretrained code models and contrastive learning baselines on both classification and pairwise metrics, improves pairwise accuracy over the strongest reproduced baseline from 0.2060 to 0.3264, and achieves pairwise accuracy comparable to that of a GPT-4o-based large language model (LLM) method at a substantially lower cost. Among the reproduced models, PatchCon also achieves the highest pairwise accuracy on test pairs from Common Vulnerabilities and Exposures (CVE) entries unseen during training and on newly disclosed vulnerabilities.

Closest papers

Nearest neighbours by meaning across the whole index.

Reassessing Dataset Quality and Fine-Tuning Practices for Vulnerability Detection on PrimeVul dataset

cs.SE ·2026-11-04 ·Norbert Sándor Szolnoki, Gábor Antal
publishedbenchmark similarity 0.76
Abstract · Presented at EASE 2026 AI Models/Data track: https://conf.researchr.org/track/ease-2026/ease-2026-ai-models---data?

Recent studies report promising results from supervised fine-tuning of pre-trained code language models for automated vulnerability detection on benchmarks such as PrimeVul. In this paper, we present a large-scale empirical evaluation of both model performance and dataset quality in this setting. We systematically fine-tune three widely used encoder models (CodeBERT, GraphCodeBERT, and UniXCoder) across five imbalance mitigation strategies, extensive hyperparameter sweeps, and multiple random seeds, resulting in 770 independent training runs. Our results reveal a consistent pattern. While standard metrics such as F1 and ROC-AUC occasionally suggest improvements, security-critical and calibration-aware measures tell a different story. Matthews Correlation Coefficient remains near zero, predicted probabilities are poorly calibrated, and evaluation under strict false-positive constraints reveals high false-negative rates. Neither imbalance handling, threshold optimization, nor post-hoc calibration recovers a meaningful discriminative signal. We also conduct a data quality assessment of the PrimeVul benchmark and identify structural issues such as minimal semantic differences between vulnerable and fixed functions, and annotation noise introduced by commit-level labeling. These findings suggests that the PrimeVul dataset, in its current form, presents significant challenges for reliable AI-based vulnerability detection in software engineering with the methodologies we used. Our study highlights the need for more robust dataset design, improved evaluation protocols, and rethinking of fine-tuning practices for security-critical tasks.

0.33

Structured Security Explanations for Learning-Based Vulnerability Detection

cs.CR cs.SE ·2026-10-27 ·Fares Trad, Yizhou Chen, Hung Viet Pham, Zeyu Sun, Gias Uddin
similarity 0.75
Abstract · 21 pages, 3 figures, 9 tables

Software vulnerability detection remains difficult because existing approaches rely on predefined analysis rules, learn vulnerability patterns implicitly from code and labels, or use large language models (LLMs) to make the final security judgment. It remains unclear whether structured descriptions of security-relevant program behavior can improve supervised detection. We present VulnLens, an explanation-augmented framework for binary C/C++ vulnerability detection that separates program-behavior analysis from the final decision. A label-blind LLM-based Explainer examines each function from seven fixed perspectives: data flow, memory access, object lifetime, arithmetic and type behavior, control flow, shared state, and external-resource access. Without receiving vulnerability labels or CWE information, it generates structured explanations. A supervised Detector combines source code and explanations through sequential cross-attention. We evaluate VulnLens on a PrimeVul-derived dataset covering ten CWE categories and test transfer to SecVulEval without retraining. On the 526-function PrimeVul test set, VulnLens achieves 75.5% accuracy and 77.2% F1, exceeding DeepSeek V4 Pro by 4.2 percentage points in accuracy and Gemini 2.5 Pro by 2.2 points in F1. Matched explanations improve F1 by 7.3 points over a separately trained code-only detector. On SecVulEval, VulnLens achieves 66.4% accuracy and 65.8% F1, outperforming DeepSeek V4 Pro by 7.6 points in accuracy and 20.0 points in F1, and Gemini 2.5 Pro by 16.6 points in accuracy and 7.8 points in F1.

0.16

Cyber-JEV: Matched Contrastive Fine-Tuning of Language Models for Fast Source-Code Vulnerability Triage

cs.CR ·2026-11-02 ·Zhuyi Zhao, Shengjie Yuan, Shengyu Wang, Kang Wei, Wang Yang +2 more
benchmarkcode/data similarity 0.74
Abstract · 8 pages, 3 figures, 5 tables. Added project page (manuscript unchanged): https://ricardoz-kisara.github.io/cyber-jev/

Single-pass scoring models such as Jev give each code function a vulnerability probability, so that security teams can rank many functions for review without generating written explanations. Training an open-weight model for this task is limited by data. Benchmarks with both vulnerable and non-vulnerable labels keep few vulnerable functions once examples related to the test data are removed, while larger public vulnerability records mostly contain vulnerable functions only. Mixing the two lets the data source predict the label, and training on the labeled data alone leaves a model that often scores a fixed function above its vulnerable original. We present Cyber-JEV, a fine-tuning method that uses fix commits to turn these records into comparison groups. Each group holds a vulnerable function, its patched version and up to two unchanged functions from the same file, so the model sees the change that removed a recorded defect. One shared model learns two tasks: classifying functions from the labeled reference data, and ranking the vulnerable function above the other members of its group. At test time the input is one function and the output is a vulnerability score. On a test benchmark of 7,270 functions, an ensemble of five fine-tuned Qwen3.5-4B models reaches an area under the receiver operating characteristic curve of 0.814, compared with 0.743 for Jev 1.13. With the 4,031 alerts of Jev as the review budget, it misses 16 of 322 vulnerable functions instead of 39. Compared with training on the reference data alone, it ranks the vulnerable version above the patched version 14.8 to 29.2 percentage points more often on three sets of real fixes. Such scores can order the changed functions of a pull request or a release for review under a fixed budget.

0.25

DeepDiff: Neuro-Dynamic Regression-Freedom Checking for Security Patch Validation

cs.CR cs.SE ·2026-10-28 ·Laboni Sarker, Shuvendu K Lahiri
datasetcode/data similarity 0.74
Abstract · 22 pages, 4 figures

Automatically validating AI-generated security patches remains a significant challenge. Eliminating a reported crash and passing existing test suites do not provide sufficient confidence that a patch correctly addresses the underlying vulnerability without introducing regressions. We present DeepDiff, a technique for regression-freedom checking that differentially evaluates candidate patches against the original vulnerable program using non-crashing inputs that exercise the affected code paths. The expectation of regression-freedom is that a correct patch should eliminate the vulnerability triggering crash while preserving the intended behavior on non-crashing inputs. To support this evaluation, we introduce a dataset of correct and incorrect patch candidates for open-source memory-safety vulnerabilities from CyberGym. We explore different approaches for non-triggering test input generation to reach the patched code, a purely neural regression-freedom checker, and a hybrid approach that combines dynamic analysis with neural techniques. Our experimental results demonstrate the effectiveness of regression-freedom checking in validating security patches while outlining the challenges that remain.

0.28

Diagnosing Generalization Boundaries in AI-Based Vulnerability Detection via Calibrated Embedding Distances

cs.SE cs.LG ·2026-10-09 ·Konstantin Ievtushenko, Ralf Lämmel, Jan Jürjens
benchmark similarity 0.72
Abstract · Submitted for review

High benchmark scores in learning-based vulnerability detection mask severe degradation under distribution shifts. This paper demonstrates that calibrated embedding geometry exposes these hidden generalization boundaries on syntactically similar vulnerable and patched code pairs. An evaluation of CodeBERT and Qwen2.5-Coder across pattern, project, and temporal shifts on a strictly paired dataset reveals three main findings. First, whitened intermediate-layer embeddings with local nearest-neighbour scoring separate unfamiliar samples with a mean out-of-distribution AUROC of 0.739 (RQ1). Second, distance scores reveal a shift-specific reliability gradient where accuracy drops by over 35 percentage points and cross-project shifts cause the sharpest degradation, even though temporal shifts drift geometrically farthest (RQ2). Third, false positives on patched code systematically correlate with geometric proximity to vulnerable training neighbourhoods, increasing from 9.4% in the lowest-risk decile to 58.8% in the highest (RQ3). These calibrated distances establish a diagnostic signal that maps representation geometry directly to model degradation without requiring auxiliary models.

0.15

Mitigating fix reversal: Counterfactual constraints for function-level vulnerability detection

cs.CR cs.AIcs.SE ·2026-11-18 ·Yun Bian, Shihao Li, Haiquan Wang, Yi Chen, Zhao Yang +1 more
similarity 0.72
Abstract · 21 pages, 9 figures

Function-level vulnerability detectors often exhibit fix reversal, assigning a higher vulnerability probability to a fixed version than to its vulnerable counterpart. Fixes usually lengthen functions, and larger length increases are associated with more severe reversal. Code-transformation experiments indicate that fix reversal mainly stems from a length shortcut: vulnerability probability increases with function length when content is fixed. We propose CFVul to address this problem. Sliding-window encoding covers complete functions, retaining fix information beyond the truncation boundary. During training, a content constraint encourages the vulnerable version to receive a higher vulnerability probability under a common length input. During inference, a length constraint averages vulnerability probabilities over fixed reference lengths, reducing the influence of actual length changes. On PrimeVul, CFVul achieves a fix-pair ranking metric, PairRank, of 59.4\%, exceeding the existing methods compared. For fixes that increase function length by more than 15\%, PairRank rises from CodeBERT's 25.8\% to 48.4\%. PairRank also improves in experiments with four additional models and three additional datasets. By weakening the length shortcut, CFVul reduces cases in which fixed versions receive higher vulnerability probabilities than vulnerable versions. It requires only existing fix pairs and applies to multiple pretrained code models, facilitating adoption in existing detectors.

0.16

VulDec: Vulnerability Detection in Binaries via Neural Decompilation

cs.CR cs.LGcs.SE ·2026-10-24 ·Francesco Crescenzo Grasso, Pietro Liguori, Roberto Natella, Domenico Cotroneo
code/data similarity 0.72
Abstract · Submitted to Automated Software Engineering

Automated vulnerability detection in binary executables is challenging due to the lack of source code and the limited semantic information available at the assembly level. Despite significant progress in machine-learning-based code analysis, the field still lacks methodologies and large, high-quality datasets that provide vulnerable code jointly at assembly- and source-level with precise vulnerability labels. To address this gap, we introduce VulDec, consisting of (i) a new large-scale, carefully curated dataset aligning real-world Assembly-C function pairs with detailed CWE annotations across multiple architectures, and (ii) the first end-to-end framework that leverages neural decompilation for binary vulnerability detection. Tuned LLM-based neural decompilers trained on our dataset achieve strong source-level recovery across architectures (Edit Similarity of approximately 56%, BLEU-4 of approximately 43%), while our tuned LLM-based detectors reach a macro F1-score of 0.61 on an in-distribution realistic test set (3% vulnerability rate). Our findings provide evidence that neural decompilation offers a powerful alternative to conventional low-level binary representations, facilitating robust binary vulnerability detection.

0.35

PatchBench: Evaluating AI Agents for Vulnerability Patching

cs.CR cs.AIcs.SE ·2026-09-03 ·Chihao Shen, Jiacheng Li, Aastha Mahajan, Jeffery Siyuan Tian, Yonghwi Kwon +1 more
benchmarkhighly cited similarity 0.71
Abstract

AI agents have recently demonstrated strong performance in automated vulnerability patching. However, existing evaluations often validate a patch only by testing whether the provided Proof-of-Concept (PoC) input still triggers a crash. This leaves two key threats to validity: agents may reproduce memorized historical developer patches, or they may generate surface-level fixes that only suppress the reported crash. We study these concerns for C/C++ vulnerability patching. We introduce a patch similarity metric to detect memorized patches. On average, 25% of the agent patches exhibit substantial similarity to historical developer patches, indicating that patch memorization is a real threat to the validity of vulnerability patching evaluations. Meanwhile, agents also frequently exploit benchmark structures to pass patch validation by patching on the crash stack trace to suppress the crash, rather than localizing and fixing the root cause of the vulnerabilities. To handle these issues, we propose PatchBench, a new benchmark for evaluating AI agents on realistic vulnerability patching tasks. PatchBench selects vulnerabilities whose ground-truth fixes lie outside the crash stack and uses vulnerability transplant and code mutations to migrate historical vulnerabilities into new repository contexts, reducing the risks of surface-level fixes and patch memorization. We develop new patch validation methods that thoroughly evaluate both security and semantic correctness of agent patches. Across 11 state-of-the-art agents, including the top three AIxCC agents, the original PoC-only validation inflates the patching task solve rate of agents by 1.83$\times$ on average. Our results reveal key limitations of current patching agents and point to future research directions for more reliable vulnerability repair.

0.41