CS Paper Radar

GraftyVul: Synthesising Insecure Programs Through Real-World Vulnerability Grafting

Omri Ram, Mitchell Horner, Ron Van der Meyden, Alsharif Abuadbba, Hammond Pearce

cs.CR cs.SE ·submitted 2026-08-28 05:10 UTC · v1
code/data

Abstract

Vulnerability datasets underpin a wide range of security research, including vulnerability detection, automated remediation, and secure code generation. However, existing datasets sacrifice at least one of three desirable properties: diversity (of language or vulnerability type), reproducibility/executability, or realism. We therefore present GraftyVul, a system that constructs vulnerable programs by grafting real-world vulnerabilities into open-source projects. This grounds the dataset in vulnerabilities observed in real-world contexts while harnessing known good build and test environments, enabling exploit-verification scripts to guarantee that an introduced vulnerability successfully alters a program's behaviour. Using GraftyVul, we generate 212 verified and exploitable vulnerable programs spanning five programming languages (Python, TypeScript, Java, Go, and C#) across 23 CWE categories. To evaluate fidelity, we introduce a language- and context-agnostic semantic embedding that compares vulnerabilities by sink, mechanism and host-feature rather than surface code. This approach outperforms standard code embeddings on cross-language clone and CWE classification. These embeddings demonstrate that GraftyVul samples retain a strong semantic signature to their source vulnerability. We additionally compare GraftyVul against 13 widely used datasets, where it attains competitive diversity while being the only reproducible-exploit dataset with broad language and CWE coverage. Finally, we illustrate GraftyVul's practical utility through an industrial case study evaluating a production vulnerability remediation system.

Comment
13 pages, 8 figures

Closest papers

Nearest neighbours by meaning across the whole index.

VICBench: A Multi-Language Benchmark for Code Vulnerability Detection

cs.CR cs.AIcs.CLcs.SE ·2026-08-12 ·Jin Lu, Xuening Han, Yang Zhong, Lin Tan, Kevin Luo +2 more
benchmark similarity 0.75
Abstract

Evaluating security vulnerability detection tools requires benchmark datasets with vulnerability-inducing commits (VICs) - the commits that first introduce vulnerabilities into codebases. VICs are essential for determining the full range of vulnerable software versions. Existing vulnerability datasets suffer from limited programming language coverage, restricted patch complexity, and narrow project scope. Through our dual annotation by human experts and an agentic workflow, we create a benchmark - VICBench - of 100 verified VICs for 100 CVEs across 88 projects in Python, Java, and C++, covering 48 CWE types. VICBench features complex real-world vulnerability fixes averaging 38.6 lines and corresponding VICs of 252.5 lines - significantly larger than prior work. Our evaluation shows that state-of-the-art algorithms V-SZZ and LLM4SZZ achieve only 33.3%-40.1% F1, confirming that using existing approaches still entails significant manual effort. VICBench enables robust evaluation of vulnerability detection approaches.

0.37

CodePoisonRAG: Knowledge Poisoning Attacks on Retrieval-Augmented Code Generation

cs.CR cs.LG ·2026-09-02 ·Varun Gadey, Ziad Marey, Alexandra Dmitrienko
similarity 0.74
Abstract · 16 pages, 1 figure. Under review

Retrieval-Augmented Code Generation (RACG) improves LLM-based software development by retrieving external code artifacts, documentation, and patches, and incorporating them into the generation context. This reliance on external knowledge introduces a critical trust boundary: poisoned artifacts can influence generated code without modifying the underlying LLM. Prior work shows that selecting existing vulnerable examples can increase the general vulnerability rate of RACG outputs, but leaves open whether a black-box attacker can construct a single task-matched artifact that propagates an attacker-selected weakness. We introduce CodePoisonRAG, a targeted upstream knowledge-poisoning framework that transforms benign fixed-code entries into poisoned artifacts. Its attack chain combines CWE-specific Vulnerability Injection, which embeds a selected source-to-sink flow while retaining task alignment, with Semantic Mislabeling, which adds false safety claims without repairing the vulnerable behavior. The attacker has no access to the victim's deployed knowledge base, retriever, re-ranker, generator, prompt, or defense mechanism and injects at most one artifact per anticipated programming task. We construct 85 poisoned artifacts covering ten CWE classes across Java and C, yielding an aggregate corpus-poisoning ratio of 0.7%. Across three generators, all 85 artifacts appear among the Top-3 results for their corresponding queries, and CodePoisonRAG achieves attack success rates between 0.80 and 0.93. Against CodeGuarder, which injects vulnerability-specific security knowledge into the generation context, the attack retains success rates between 0.40 and 0.71. These results show that RACG poisoning extends beyond the incidental propagation of existing vulnerabilities to the targeted construction and propagation of attacker-selected weaknesses.

0.42

Finding Vulnerabilities via LLM-Augmented Semantics-Aware Type-Checking

cs.CR ·2026-08-14 ·Ruizhe Wang, Meng Xu, N. Asokan
similarity 0.73
Abstract

Vulnerability detection via static analysis traditionally relies on security experts encoding insecure coding patterns into algorithmic rules. However, this approach often focuses on syntactic patterns and overlooks deeper semantic information in the code, such as the meanings of variable and function names. As software systems grow more complex, modeling vulnerabilities using only syntactic rules becomes increasingly challenging. In this paper, we propose a semantics-aware approach to detecting software vulnerabilities. We present SETYPE, a semantics-aware type system that can be derived directly from source code based solely on the meanings of symbols and expressions in natural language. In the SETYPE type system, both type inference and checking are performed by Large Language Models (LLMs), and a failed type check indicates a potential vulnerability. We prototype PYSETYPE to demonstrate the feasibility of SETYPE for detecting vulnerabilities in Python web applications. Our evaluation on real-world applications achieves 87% detection precision and 88% detection accuracy. Using PYSETYPE, we identified 15 potential zero-day vulnerabilities, nine of which were confirmed by developers.

0.04

Beware EviLLM: Enabling Vulnerability Injection via Large Language Models

cs.CR cs.AI ·2026-10-02 ·Zeezoo Ryu, Simon Chung, Muhammad Faraz Karim, Anna Raymaker, Karan Singh Jodha +2 more
similarity 0.73
Abstract

Advances in large language models (LLMs) have enabled AI-driven code generation from natural language specifications, introducing new attack surfaces for injecting vulnerabilities into software. Prior work has studied this problem only in benign settings where vulnerabilities are introduced inadvertently, or under unconventional threat models where the LLM itself is malicious (backdooring) or the user is the attacker (jailbreaking). In this paper, we study a more realistic threat model: a third-party adversary, with capabilities comparable to existing cybercriminals, compromises the AI code generation pipeline to deliberately introduce vulnerabilities. We call this the EviLLM attack. We have implemented two instances of EviLLM, each of which only requires the underlying LLM to be accessed through a compromised account or browser, and can inject vulnerabilities from 13 CWE classes. As we show in our feasibility study, both attack vectors are already used to implement many existing cyberattacks. Our user study shows that 7 out of 8 and 10 out of 13 participants did not notice the vulnerabilities injected by the two instances of EviLLM, and 13 out of 21 participants "rarely" or "never" considered the risk of an attack like EviLLM.

0.15

Combining LLM and Dynamic Analysis for Universal Testing of Static Application Security Testing Tools

cs.CR cs.SE ·2026-10-15 ·Huaien Zhang, Yu Pei, Hao Chen, Shin Hwei Tan
similarity 0.72
Abstract

Static application security testing (SAST) tools help detect potential software vulnerabilities, but they themselves contain faults undermining reliability and usability. Automated testing can address these issues, yet current techniques suffer from limited program transformation diversity and lack mechanisms to verify generated variants, leading to reduced fault detection and high manual effort. We present LLM4SAST, the first universal framework detecting faults in SAST tools using large language models and dynamic feedback. Our key insight is leveraging LLMs to understand and transform code in diverse programming languages. We design novel metamorphic relations with transformations to guide variant generation, using an LLM to produce multi-language variants that satisfy these relations. To reduce erroneous variants and identify unique faults, we propose two heuristics: dynamic-feedback-driven selection and program-differencing-based clustering. For each metamorphic relation, a report-centric test oracle detects variants that fail tests and identifies underlying faults by differentially analyzing SAST reports of the seed program and its variants. LLM4SAST identified 66 unique faults across six SAST tools, with 32 confirmed or fixed after being reported to developers, demonstrating effectiveness. The proposed heuristics eliminate 47.8% of false positives, and LLM4SAST detects 3.4 times more faults while requiring only 39.8% of the variants compared to existing approaches.

0.19

SEEK: Agentic Knowledge Graph Retrieval for Secure Code Generation

cs.CR cs.LG ·2026-10-10 ·Varun Gadey, Markus Tohn, Lichao Wu, Ahmad-Reza Sadeghi, Alexandra Dmitrienko
code/data similarity 0.72
Abstract · 21 Pages, 3 figures, Under revision

Large Language Models (LLMs) can generate functionally correct code while introducing exploitable security weaknesses, particularly when developers specify functionality without explicit security requirements. Recent state-of-the-art approaches mitigate this problem using external security knowledge, but largely rely on flat similarity-based retrieval over independent artifacts, making it difficult to infer the relevant vulnerability space and retrieve targeted knowledge when direct matches are unavailable. We propose SEEK, an agentic Graph RAG framework for secure code generation that constructs a security knowledge graph using the Common Weakness Enumeration (CWE) taxonomy as its structural backbone and injects it with curated real-world CVE vulnerability-fix instances. At inference time, a Planner Agent predicts potential CWE weaknesses associated with the developer query, which guide Graph Scoping to restrict retrieval to relevant regions of the CWE taxonomy before deterministic multi-faceted candidate ranking. A Selection Agent subsequently validates the predicted weaknesses and retrieved candidates prior to code generation. Across five open-source LLMs, we evaluate SEEK in an out-of-distribution setting on CodeGuard+ and CyberSecEval, spanning C, C++, and Python. SEEK achieves the highest SecurePass@1 of 77.45% on CodeGuard+ and the highest aggregate ICD pass rate of 79.45% on CyberSecEval, surpassing prior approaches. Moreover, SEEK achieves 85.2% SecurePass@1 across 13 unseen CWE types with no directly linked vulnerability-fix instances, demonstrating strong generalizability, while maintaining acceptable inference latency costs.

0.31

Structured Security Explanations for Learning-Based Vulnerability Detection

cs.CR cs.SE ·2026-10-27 ·Fares Trad, Yizhou Chen, Hung Viet Pham, Zeyu Sun, Gias Uddin
similarity 0.71
Abstract · 21 pages, 3 figures, 9 tables

Software vulnerability detection remains difficult because existing approaches rely on predefined analysis rules, learn vulnerability patterns implicitly from code and labels, or use large language models (LLMs) to make the final security judgment. It remains unclear whether structured descriptions of security-relevant program behavior can improve supervised detection. We present VulnLens, an explanation-augmented framework for binary C/C++ vulnerability detection that separates program-behavior analysis from the final decision. A label-blind LLM-based Explainer examines each function from seven fixed perspectives: data flow, memory access, object lifetime, arithmetic and type behavior, control flow, shared state, and external-resource access. Without receiving vulnerability labels or CWE information, it generates structured explanations. A supervised Detector combines source code and explanations through sequential cross-attention. We evaluate VulnLens on a PrimeVul-derived dataset covering ten CWE categories and test transfer to SecVulEval without retraining. On the 526-function PrimeVul test set, VulnLens achieves 75.5% accuracy and 77.2% F1, exceeding DeepSeek V4 Pro by 4.2 percentage points in accuracy and Gemini 2.5 Pro by 2.2 points in F1. Matched explanations improve F1 by 7.3 points over a separately trained code-only detector. On SecVulEval, VulnLens achieves 66.4% accuracy and 65.8% F1, outperforming DeepSeek V4 Pro by 7.6 points in accuracy and 20.0 points in F1, and Gemini 2.5 Pro by 16.6 points in accuracy and 7.8 points in F1.

0.16

SemVul: Semantic-Enhanced Graph Neural Networks for Code Property Graph-based Vulnerability Detection

cs.SE cs.CR ·2026-09-05 ·Younas Affan, Leonardo Regano, Giorgio Giacinto
publishedbenchmark similarity 0.71
Abstract

Vulnerabilities in source code are often the root cause of cyberattacks worldwide, as attackers exploit weaknesses in software to gain unauthorized access, steal data, or disrupt services. In this study, we evaluated existing research approaches and propose SemVul, a vulnerability detection pipeline that demonstrates better generalization and higher accuracy in learning vulnerable code patterns. We propose a Code Property Graph-based vulnerability-detection approach combined with semantic-level enhancement, enabling the model to capture both the program's structural flow and the semantic meaning of the code. Our approach integrates both node-level and edge-level semantic embeddings using pre-trained code embedding techniques. We systematically evaluate multiple GNN architectures on publicly available benchmark datasets. SemVul is generic with respect to the programming language and supports multiple architectures. By integrating structural and semantic information, the proposed approach improves vulnerability detection performance. Our results show that SemVul outperforms existing approaches and provides better generalization.

0.33