CS Paper Radar

A First Look at System One Decision Models for Malware Detection

Pier Mattia Basciano, Giovanni Farina, Salvatore Monteleone

cs.CR cs.AIcs.LG ·submitted 2026-11-16 08:08 UTC · v1

Abstract

Malware detection needs scores calibrated at a fixed false positive rate (FPR) and decisions an analyst can read: feature-based detectors give the first, large language models (LLMs) on decompiled code the second, through generated text of uncalibrated confidence. System One decision models answer a typed question about a text with a probability over its declared answers in one forward pass. We present the first evaluation of these models for Windows Portable Executable malware detection, analyze how they decide, and propose S1Mal, an ensemble on the typed answers and the embedding of CLM-8B. In an isolated virtual machine, Ghidra and capa decompose each binary, and the host renders the records into a canonical textual state, a digest of headers, imports, MITRE ATT&CK capabilities and strings, optionally with decompiled functions, without class shortcuts. Three readouts score each model on the state: A0, the probability of one verdict question; A1, 32 atomic behavioral questions weighed by gradient-boosted trees; A2, heads on the model's state embedding. S1Mal feeds gradient-boosted trees with the atomic answers and the A2 score of CLM-8B. We evaluate CLM-8B, ten open decision models and two LLMs read through logits on BODMAS temporal splits, SOREL-20M out-of-distribution malware and 14,534 matched benign files. On BODMAS, A0 wastes the static evidence: its area under the receiver operating characteristic curve (AUROC) spans 0.4025-0.8590 across 13 models. A1 recovers it, raising every model's AUROC by +54.61% on average; A2 lifts CLM-8B by +0.2749 to 0.9943, above a combiner on digest facts (0.9901), and reaches 0.9853-0.9948 on nine open models. S1Mal reaches 0.9946 and detects 93.47% of the malware at a realized FPR of 1.45%, and 59.15% at 0.1% FPR on the test ROC curve, against 46.59% with A2. S1Mal gives analysts calibrated decisions, each with the atomic answers behind it.

Closest papers

Nearest neighbours by meaning across the whole index.

TRIAGE-CPG: A Cross-Model Study of Budgeted Static Malware Triage with Large Language Models

cs.CR cs.AI ·2026-11-16 ·Paul-Andrei Sava, Alexander Küchler, Daniel Kowatsch, Bernhard Grill, Jennifer Pullman +2 more
published similarity 0.67
Abstract · Code and artifacts available here: https://doi.org/10.5281/zenodo.22771897

When triaging unknown binaries, security analysts must decide which samples to clear as benign, flag as malicious, or escalate for deeper analysis. Large Language Models (LLMs) could support these decisions by interpreting decompiled code, but a binary can contain hundreds or thousands of functions, making exhaustive LLM analysis impractical. We study how current LLMs behave when static triage is formulated as budgeted evidence acquisition: which functions they inspect and how they trade off benign clearance, malware detection, and escalation. To enable a controlled comparison, we implement TRIAGE-CPG, a fixed pipeline over a code property graph (CPG) of a decompiled binary. An LLM-free heuristic selects seed functions, an agent expands the resulting evidence dossier under a fixed budget, and a reporting stage produces a benign, suspicious, or malicious verdict together with a behavioral report. We evaluate eight closed-source and open-weight models on 1,000 Windows PE binaries, using a 600-sample test split and the same zero-shot protocol for every model. We assess the ternary verdicts under both a strict binary reading and triage balanced accuracy (tBA), which credits malware that is flagged or escalated and benign samples only when cleared. After analyzing about 40 of a mean 1,000 functions per binary, the highest-scoring model reaches a strict balanced accuracy of 0.805 and a tBA of 0.855, flagging 65% of malware, escalating another 12%, and clearing 94% of benign samples. The models implement different triage policies: high-specificity models produce small, malware-rich escalation queues, whereas recall-oriented models escalate or flag much more broadly. Both closed-source and open-weight models appear across this spectrum. Our results identify no universal deployment configuration: model choice itself determines the clear/flag/escalate trade-off under limited static evidence.

0.41

Drift-Aware Explainable Prioritization for Windows PE Malware Forensic Triage with Semantic Consistency Checks

cs.CR ·2026-11-18 ·Mohammed A. Alharbi, Toqeer Ali Syed
similarity 0.66
Abstract

Forensic investigators must decide which suspicious files to examine first. Machine-learning (ML) and deep-learning (DL) malware detectors usually return a bare score that neither explains why a file deserves attention nor signals when changing data make the model unreliable. We present a framework for Windows Portable Executable (PE) malware forensic triage that works only on pre-extracted static features and never executes malware. A calibrated maliciousness score sets each file's triage priority, and a per-sample feature attribution explains the prediction. A per-sample consistency score measures whether the top attributed features agree with documented behaviors from the Malware Behavior Catalog (MBC) and MITRE Adversarial Tactics, Techniques & Common Knowledge (ATT&CK), where the available features support such interpretation. Conformal evaluation and statistical distribution tests flag drift. Each file receives one investigator-oriented triage output. The evaluation protocol, specified before testing, uses a time-ordered split of EMBER2024 as the primary dataset and BODMAS for family disjoint checks, and compares every component with simple baselines on predictive performance, prioritization effectiveness, explanation quality and robustness over time.

0.25

Breaking the Black Box: Byte-Level Boundary Inference of Real-World Antivirus Systems

cs.CR ·2026-09-25 ·Jieshuai Yang, Zhi Wang, Yan Jia, Zhenhua Wu, Jianfei Tang +4 more
similarity 0.65
Abstract

Existing approaches for understanding the detection logic of real-world antivirus (AV) software infer only binary malware/benign decisions from black-box queries, providing limited insight into the fine-grained decision-critical regions that govern AV detection. In this paper, we present \textbf{AVHunter}, the first framework for inferring byte-level decision-critical regions of real-world AV products under a black-box threat model. AVHunter constructs the first large-scale Byte-Level AV Boundary Dataset (BABD) by systematically probing 11 real-world AV products, revealing that modern AV detections are largely associated with a small number of compact decision-critical byte regions. Leveraging BABD, AVHunter trains AV-specific models that not only reproduce binary AV decisions, but also localize the decision-critical byte regions underlying these decisions, achieving an average boundary prediction recall of 85.07% while maintaining 97.43% detection agreement with the target AVs. We further validate that the predicted regions capture genuine AV decision knowledge through boundary-guided malware evasion, false-positive induction on benign executables, and a seven-month longitudinal study demonstrating that the inferred regions remain largely stable as AV products evolve. Overall, AVHunter moves beyond conventional binary-label AV modeling by enabling fine-grained boundary-region localization and revealing a new form of AV knowledge leakage with important implications for malware analysis, AV security, and boundary-aware defenses.

0.13

ALIBI: Adversarial Legitimacy Injection in Binary Input against LLM Malware Analyzers

cs.CR cs.LG ·2026-09-17 ·Hyeongjun Choi, Wonyoung Jung, Haehoon Seo, Sungyup Nam
similarity 0.64
Abstract · 12 pages, 4 figures, 4 tables

Large language models are being integrated into malware triage workflows as reasoning components that summarize static evidence and produce analyst-facing verdicts. This paper shows that the same reasoning capability introduces a new attack surface. We present ALIBI, a semantic cover story attack against frontier LLM-based malware analyzers. ALIBI adds a small, non-executed read-only section to a compiled binary, containing a coherent but false security product narrative, without altering imports or executable behavior. Instead of issuing direct instructions to the model, it reframes suspicious evidence as expected behavior of a benign endpoint security tool. On a frozen PE set of 50 malicious samples, the payload flips 30 of the 35 baseline-malicious samples to benign on Gemini 2.5 Pro, while GPT-5.5 Pro and Claude Opus 4.7 produce substantial severity downgrades with significant confidence reductions even when verdict labels are preserved. The attack transfers to ELF binaries, where Gemini flips 16 of 40. A verification-guided defense prompt roughly halves the benign verdicts, but 42.9 percent of malicious samples still reach benign. LLM malware analyzers therefore require provenance checks that separate verified facts from attacker-controlled claims, not narrative trust.

0.49

Rethinking Neural Decompilation: Candidate Selection as the Hidden Bottleneck

cs.CR cs.AI ·2026-11-02 ·Felix Mächtle, Gautham Sriram, Manjil Neupane, Nils Loose, Thomas Eisenbarth
publishedcode/data similarity 0.64
Abstract · Published at the International Conference on Information and Communications Security (ICICS 2026). Code: https://github.com/UzL-ITS/rethinki…

Reverse engineering obfuscated binaries is a core task in malware analysis and software security auditing. While neural decompilers based on Large Language Models (LLMs) have improved the readability and functional fidelity of recovered code over heuristic tools such as Ghidra and IDA Pro, a substantial gap remains between best-sample performance and practical single-output accuracy, particularly under code obfuscation. We study this gap through the lens of candidate generation and candidate selection. Using the LLM4Decompile architecture, we show that ground-truth tokens lie within the model's top-five next-token predictions in 95.6% of positions on average and that multi-sample decoding reaches a pass@100 of 93% on HumanEval-Decompile, indicating that correct reconstructions are typically already present in the model's sampling space. We then evaluate a generate-then-select pipeline that ranks 100 candidates per input with text-, syntax-, embedding-, consensus-, and likelihood-based selectors. Negative perplexity emerges as the strongest static selector on HumanEval-Decompile (73.3% pass@1), while execution-guided selection nearly closes the oracle gap on ExeBench (86.3% pass@1). Combined with one epoch of fine-tuning on our newly released ExeBench-Obfuscated dataset containing 362,647 obfuscated OLLVM samples, the same pipeline increases the pass@1 rate for obfuscated binaries to 88.8%, directly contradicting prior claims that released LLM4Decompile checkpoints are unsuitable for obfuscated inputs.

0.36

SCRIPTIOC-BENCH: A Benchmark for Recognizing Actionable Threat Intelligence from Script-Based Malware using LLMs

cs.CR cs.AI ·2026-09-05 ·Hanna Kim, Jian Cui, Minkyoo Song, Hwanjo Heo, Seungwon Shin +2 more
benchmark similarity 0.63
Abstract · 22 pages

Script-based malware remains a prevalent attack technique. These scripts often contain indicators of compromise (IOCs) that provide actionable threat intelligence. However, statically recovering such indicators is challenging, as relevant values may be dispersed or transformed within code. Although large language models (LLMs) have shown promise in security analysis, their ability to recover IOCs from malicious scripts remains underexplored. We present SCRIPTIOC-BENCH, a benchmark for measuring static IOC extraction capability on real-world malicious scripts. The benchmark comprises 634 manually verified JavaScript, PowerShell, and VBScript malware samples covering four IOC types (URLs, domains, IP addresses, and filesystem artifacts). We further stratify ground-truth IOCs by recovery level, distinguishing directly exposed indicators from those requiring decoding or reconstruction. Using this benchmark, we evaluate a broad range of proprietary and open-weight LLMs and show that IOC recovery without execution remains challenging across model scales: the strongest model reaches only 65.4 F1. To characterize how recovery fails, we introduce a false-positive taxonomy and use it to compare the error profiles of the evaluated models. We further study two mitigations on a small open-weight model, deterministic string utilities and task-specific adaptation, finding that they provide complementary recovery gains, raise precision, and shift errors toward sample-grounded mismatches.

0.32

MARS: Malware Analysis with Rule-Based Scoring of LLM Claims

cs.CR cs.AI ·2026-10-07 ·Hyeongjun Choi
similarity 0.63
Abstract · 17 pages, 3 figures, 12 tables

Large language models can triage malware through direct verdicts or behavioral claims scored by an external policy. We present MARS, a malware triage framework, and compare direct classification with single-pass claim scoring using the same evidence collector and identical static evidence bundles for each model. The evaluation covers 1,195 PE and ELF binaries grouped into 1,001 near-duplicate clusters and six language models, with deterministic rules providing a baseline. Direct classification is more accurate for all six models. On samples with usable outputs from both paths, its accuracy advantage ranges from 3.7 to 20.9 percentage points, with all 95% cluster-bootstrap confidence intervals for the differences above zero. It also achieves higher malicious alert recall in ten of twelve platform and model combinations. Claim mediation provides no consistent reduction in performance variation across models. Separate subset studies find more consistent alert decisions for direct classification and a larger recall loss for the claim path when predefined indicator fields are removed. In a family identification probe, claims yield higher accuracy than verdict labels but lower accuracy than evidence text. Retained claims expose the inputs to verdict computation and permit policy revision without another model call. We reproduce archived verdicts exactly and apply a revised policy to the same records, including outputs from two additional models withdrawn by their provider. Under the evaluated claim taxonomy and additive policy, these results favor direct classification when only a verdict is required, while demonstrating that retained claims support explicit policy inspection and revision.

0.45

Delphi Scanner: efficient and interpretable static malware detection via API sequence modeling

cs.CR cs.LG ·2026-09-17 ·Bijied Brahimi, Vincent Cohadon, Gabriel Glazman, Rayan Al Mohaize, Omran Berjawi +1 more
similarity 0.63
Abstract

Static malware detection for Windows Portable Executable files demands a careful balance between detection effectiveness, computational efficiency, and analytical interpretability. This paper introduces Delphi Scanner, a static malware detection system for Windows PE files that balances efficiency with behavioral interpretation. It uses a convolutional neural network (CNN) to model Windows API sequences to classify PE and a decoupled interpretation layer based on a rule-based layer to categorize APIs into high-level malicious capabilities. Evaluated on over 190,000 Windows PE files, the system achieves 95.35% accuracy with a 1.53~MB model footprint. Robustness experiments on 5,647 out-of-distribution MalwareBazaar samples, paired packed and unpacked executables, and three adversarial manipulation strategies confirm generalization beyond the training distribution and resistance to functionality-preserving evasion techniques. Overall, these results demonstrate that API sequence-based static analysis offers a practical, interpretable, and efficient foundation for malware triage in local deployment scenarios.

0.22