CS Paper Radar

CASHEWS: Source Preprocessor for LLM-based Malicious Package Detection

Jean-Charles Noirot Ferrand, David Adei, Anders Møller, Alexandros Kapravelos

cs.CR ·submitted 2026-09-16 16:00 UTC · v1

Abstract

Malicious npm package detection tools now leverage LLMs' semantic understanding of source code to detect malicious intent at scale. This capability has proven invaluable in identifying packages involved in recent supply-chain attacks such as Shai-Hulud. However, threat actors exploit the limited context windows of LLMs through JavaScript techniques such as code obfuscation that yields high token density and bundling malicious code with benign packages, causing detectors to skip large files or miss malicious behavior. This creates an attack surface for evading detection. In this paper, we present CASHEWS, a JavaScript preprocessor that reduces file size by rewriting source code to remove code that is irrelevant to analysis or likely to mislead the model. Given a package source file, CASHEWS deobfuscates it through iterative decoding, extracts bundled modules and dynamically executed code, identifies malicious sinks and computes backward slices that reach them, and abbreviates long literals and identifiers to produce a compact representation for the detector. Across 512 large package files, two scanner types, and three LLMs, CASHEWS increases analysis coverage from 69.1--85.7% to 98.8--100% and reduces the false-negative rate by up to 18.6 percentage points. CASHEWS also has a median preprocessing time of 30 seconds while reducing net analysis cost by 34.6%, making registry-wide LLM-based analysis more practical. By preprocessing source code before analysis, CASHEWS enables researchers and industry practitioners to use more powerful models for malicious package detection at the same or lower analysis cost as less powerful models.

Closest papers

Nearest neighbours by meaning across the whole index.

DistillGuard: Malicious NPM Package Detection and API Attack Chain Analysis via Static Graph and LLM Distillation

cs.CR ·2026-09-24 ·Siyuan Pang, Yepeng Yao, Zhengwei Jiang, Zijing Fan, Baoxu Liu
code/data similarity 0.71
Abstract

The Node.js ecosystem heavily relies on NPM packages, and software supply chain attacks targeting malicious NPM packages are rampant. Malicious code primarily triggers during package installation, import, and runtime. Traditional static analysis fails to understand code semantics; machine learning-based methods rely on feature extraction, which suffers from concept drift; existing LLM solutions suffer from high invocation costs, high data security risks, and poor performance. To overcome these limitations, we propose DistillGuard, a lightweight malicious NPM package detection framework that combines static graph semantic analysis and LLM knowledge distillation. DistillGuard first acquires multi-granular features through three static analysis modules. Then, it leverages online LLM to distill high-quality security knowledge and structured labels. Finally, it uses LoRa to efficiently fine-tune the open-source Qwen3-8B model to support offline deployment. Experiments show that DistillGuard achieves an accuracy of 95.3\%, a precision of 99.4\%, and an F1 score of 93.8\%, outperforming state-of-the-art tools, improving the F1-score by 11.1 to 30.0 percentage points over the baselines. Our empirical research further reveals the stages of malicious attacks and the distribution of malicious behaviors. We also summarized eight typical API attack chains for malicious activities, providing practical insights for NPM supply chain security defense.

0.18

VMSpec: A Decompiler-based Approach against Virtual Machine Obfuscation

cs.CR ·2026-10-21 ·Fabian Kilger, Leonard Auer, Manuel Andreas
benchmarkhot direction similarity 0.69
Abstract

Malicious actors frequently obfuscate their code to hinder investigative reverse engineering efforts. Among the most formidable techniques is virtual machine (VM) obfuscation: programs are translated into bytecode for a custom VM, often with a randomized virtual instruction set and distributed alongside a corresponding interpreter. Manual analysis and deobfuscation scale poorly due to the complexity of VM obfuscation and the diversification techniques that render each VM and interpreter unique. Existing methods primarily focus on simplifying execution traces and identifying virtual execution boundaries. However, comparatively little attention has been given to recovering higher-level program semantics from VM-obfuscated code. To address this gap, we propose VMSpec, a novel decompiler-based approach that automates the construction of VM processor specifications, enabling existing, well-known decompilers to analyze the VM-obfuscated code. We evaluate our prototype of VMSpec on 147 programs from an established obfuscation benchmark, comprising hash functions, standard algorithms, and randomized functions, obfuscated with different configurations of the Tigress and Code Virtualizer obfuscators. To demonstrate real-world applicability, we additionally include six domain generation algorithms stemming from real-world malware. VMSpec completely strips VM-obfuscation from nearly all obfuscated samples, producing output comparable to Ghidra's decompilation of the original code. Furthermore, it is the first method capable of reconstructing the complete program behavior of VM-obfuscated programs with infinite, input-dependent paths and recursive calls.

0.23

CS-Guard: Benchmarking LLM Guardrails for Code Generation Security

cs.CR cs.AI ·2026-09-09 ·Jinyang Li, Mingyu Guo, Hung X. Nguyen
benchmark similarity 0.69
Abstract

Large language models (LLMs) have been ex- ploited to generate malware, but the effective- ness of guardrails for code generation secu- rity remains unclear. We introduce CS-Guard, the first benchmark to systematically evalu- ate guardrails for code generation security. It covers 1) text-to-code generation with 1000 high-quality malware-generation prompts, 7 jailbreak attacks, and a novel fictional scenario attack (FSA) that embeds malicious intent in a legitimate fictional software-development sce- nario; and 2) code-to-code generation with 331 code prompts spanning code infilling, code completion, and code translation. We empiri- cally evaluate 9 guardrails across seven LLMs. We find that current guardrails perform poorly against malicious code-generation re- quests: for text-to-code, the average attack success rate (ASR) after jailbreaks reaches about 50% for many guardrails; for code-to- code, average ASR approaches 100% on base LLMs and remains high across many guardrails (14.4% to nearly 100%). Our FSA also achieves ASR close to 100% across many guardrails, raising major reliability concerns for real-world software development. To sup- port future research, CS-Guard uses a modular three-layer guardrail taxonomy that lets devel- opers register guardrails for evaluation. We release the benchmark and data to enable fur- ther community evaluation.

0.14

WebMaze: Defending Against LLM-Powered Web Scanners with Functionality-Coupled Deception

cs.CR ·2026-10-28 ·Pengfei Zeng, Changhua Luo, Penghui Li
similarity 0.69
Abstract · 10 pages, 4 figures

Large language models (LLMs) have enabled more effective vulnerability scanning of web applications. However, these capabilities also make it significantly easier for attackers to automate cyberattacks on the web. In this work, we investigate how to divert malicious LLM-based web scanning, with preservation of genuine functionality as a deployment requirement. We systematically study six LLM-based scanners and summarize how they gather information and iteratively select actions based on observations. Guided by the study, we then present WebMaze, a defense that preserves genuine application functionality while diverting LLM-based scanners to a generated decoy world. WebMaze embeds functionality-coupled traps in genuine pages and constructs view-tailored interactive decoy worlds from the application's existing views. The traps blend into genuine components and expose entry points to decoy worlds that sustain exploration through connected pages and interactions without changing genuine functionality. Across 12 web applications and three LLM-based scanners under the default no-disclosure setting, decoy pages account for an average of 93.0% of the distinct pages requested per run. This demonstrates that WebMaze effectively diverts scanners away from genuine application pages and into the generated decoy worlds. Even when explicitly informed about the decoy mechanism and instructed to avoid decoy pages, scanners continue to explore predominantly decoy pages.

0.21

Exploring Automated Vulnerability Identification in JavaScript Code Using Large Language Models

cs.CR cs.AI ·2026-09-12 ·Manit Kaushik, Ishir Bhardwaj, Pranav Gupta, Pankaj Jalote, Arun Balaji Buduru
similarity 0.69
Abstract · 8 pages, 1 figure, 4 tables

JavaScript powers approximately 98.8% of all websites, making vulnerabilities in its code a significant security risk, yet existing detection approaches such as Static Application Security Testing (SAST) tools often fail to identify many real-world vulnerabilities when applied to isolated code snippets. This paper presents an empirical study of Large Language Model (LLM)-based vulnerability identification for JavaScript programs, evaluating three LLM families (Gemini 1.5 Flash, GPT-4o Mini, DeepSeek-R1-Distill-Llama-8B) across multiple prompting strategies (zero-shot, chain-of-thought, few-shot) and fine-tuning approaches on a dataset of 1,125 JavaScript code snippets spanning five Common Weakness Enumeration (CWE) categories: Injection (CWE-74), OS Command Injection (CWE-78), Cross-Site Scripting (CWE-79), SQL Injection (CWE-89), and Uncontrolled Resource Consumption (CWE-400). Our experiments show that LLMs substantially outperform traditional SAST tools on snippet-level vulnerability identification, with a fine-tuned Gemini 1.5 Flash model achieving 60% detection accuracy compared to near-zero performance from rule-based analyzers. We find that fine-tuning improves accuracy from 29% to 60%, Chain-of-Thought prompting benefits reasoning-capable models such as GPT-4o Mini, few-shot prompting is effective for polymorphic vulnerabilities such as Cross-Site Scripting, and performance varies across vulnerability categories, reaching up to 84% accuracy for structured vulnerabilities such as SQL Injection. These results indicate that LLMs provide a practical approach for automated vulnerability identification in JavaScript code, particularly when combined with task-aligned supervision, though they should complement rather than replace existing security analysis workflows due to limited recall and uneven performance across vulnerability types.

0.18

TRIAGE-CPG: A Cross-Model Study of Budgeted Static Malware Triage with Large Language Models

cs.CR cs.AI ·2026-11-16 ·Paul-Andrei Sava, Alexander Küchler, Daniel Kowatsch, Bernhard Grill, Jennifer Pullman +2 more
published similarity 0.68
Abstract · Code and artifacts available here: https://doi.org/10.5281/zenodo.22771897

When triaging unknown binaries, security analysts must decide which samples to clear as benign, flag as malicious, or escalate for deeper analysis. Large Language Models (LLMs) could support these decisions by interpreting decompiled code, but a binary can contain hundreds or thousands of functions, making exhaustive LLM analysis impractical. We study how current LLMs behave when static triage is formulated as budgeted evidence acquisition: which functions they inspect and how they trade off benign clearance, malware detection, and escalation. To enable a controlled comparison, we implement TRIAGE-CPG, a fixed pipeline over a code property graph (CPG) of a decompiled binary. An LLM-free heuristic selects seed functions, an agent expands the resulting evidence dossier under a fixed budget, and a reporting stage produces a benign, suspicious, or malicious verdict together with a behavioral report. We evaluate eight closed-source and open-weight models on 1,000 Windows PE binaries, using a 600-sample test split and the same zero-shot protocol for every model. We assess the ternary verdicts under both a strict binary reading and triage balanced accuracy (tBA), which credits malware that is flagged or escalated and benign samples only when cleared. After analyzing about 40 of a mean 1,000 functions per binary, the highest-scoring model reaches a strict balanced accuracy of 0.805 and a tBA of 0.855, flagging 65% of malware, escalating another 12%, and clearing 94% of benign samples. The models implement different triage policies: high-specificity models produce small, malware-rich escalation queues, whereas recall-oriented models escalate or flag much more broadly. Both closed-source and open-weight models appear across this spectrum. Our results identify no universal deployment configuration: model choice itself determines the clear/flag/escalate trade-off under limited static evidence.

0.38

Sign Here: Attributing a Source for a More Trustworthy Software Supply Chain

cs.CR cs.SE ·2026-10-20 ·Derek Garcia
code/data similarity 0.67
Abstract

Software supply chain attacks continue to be on the rise, exploiting the trust that helps open source ecosystems thrive. Recent incidents like the Shai-Hulud worm and the Axios compromise demonstrate that a single malicious package can have wide reaching and devastating impacts overnight. These high profile attacks can be traced back to the same culprit: package registries. Although they are a useful convenience to pull prebuilt packages from, is such a commonly exploited component of the software supply chain worth the risk? In this paper, we investigate whether removing the registry from the developer workflow by building packages directly from source can serve as a malware prevention strategy - effectively removing an entire attack surface. Our hypothesis is that malicious packages avoid attributing a source to minimize visibility, while benign packages seek exposure and therefore link to their repositories. We evaluate this through three research questions: how effective is building from source at avoiding malware, how feasible is it to build from source in the current state of the supply chain, and how the building from source strategy compares to state-of-the-art security scanners. To do so, we (1) construct an aggregate dataset of 47,066 malicious packages from four academic and one industry malicious datasets source across npm and PyPI ecosystems, (2) build a snapshot index of all 3.9 million npm and 764 thousand PyPI packages, and (3) analyze all malicious samples alongside a statistically significant sample of 16,569 packages from each registry, totaling 33,138 packages. We find that simply checking for a source has a success rate between 87.7 and 99.9 percent of preventing malware, depending on the level of source vetting. Consequently, there is a significant lack of transparency within benign packages, with only 47.4 percent attributing a source an end user could build from.

0.20

Beware EviLLM: Enabling Vulnerability Injection via Large Language Models

cs.CR cs.AI ·2026-10-02 ·Zeezoo Ryu, Simon Chung, Muhammad Faraz Karim, Anna Raymaker, Karan Singh Jodha +2 more
similarity 0.67
Abstract

Advances in large language models (LLMs) have enabled AI-driven code generation from natural language specifications, introducing new attack surfaces for injecting vulnerabilities into software. Prior work has studied this problem only in benign settings where vulnerabilities are introduced inadvertently, or under unconventional threat models where the LLM itself is malicious (backdooring) or the user is the attacker (jailbreaking). In this paper, we study a more realistic threat model: a third-party adversary, with capabilities comparable to existing cybercriminals, compromises the AI code generation pipeline to deliberately introduce vulnerabilities. We call this the EviLLM attack. We have implemented two instances of EviLLM, each of which only requires the underlying LLM to be accessed through a compromised account or browser, and can inject vulnerabilities from 13 CWE classes. As we show in our feasibility study, both attack vectors are already used to implement many existing cyberattacks. Our user study shows that 7 out of 8 and 10 out of 13 participants did not notice the vulnerabilities injected by the two instances of EviLLM, and 13 out of 21 participants "rarely" or "never" considered the risk of an attack like EviLLM.

0.15