CS Paper Radar

Stress-Testing Structure-Aware Calibration of Malware Graph Neural Networks under Type Shift

Junru Zhu, Yixin Yang, Xiaoqing Ding, Ruoyu Qi

cs.CR ·submitted 2026-09-22 00:00 UTC · v2

Abstract

Post-hoc malware calibrators can condition confidence on graph structure, but their structural inputs may leave the support represented by validation data under malware-type shift. We study this risk on MalNet-Tiny by holding out each of four malware types across three seeds, freezing a graph isomorphism network, and fitting post-hoc mappings only on known-type data. On known-type tests, generic and combined structural calibrators have negative log likelihood (NLL) of 0.4341 and 0.4628. Their mean held-out-type gap widens to 0.1369, with a four-type 95% interval of [-0.0125, 0.2864] and improvement in one of four types. Eight label-independent nuisance variables produce a smaller mean NLL increase of 0.0415, but the paired combined-versus-nuisance interval includes zero. We then use calibration data to define a community-support guard: predictions outside the 95th percentile of calibration-standardized community displacement revert to the generic calibrator. The guard reduces combined NLL by 0.1133 and high-confidence errors from 43.25 to 33.25 per 400-sample cell. It also outperforms matched-rate random fallback on the original folds, while generic and uncalibrated probabilities retain lower held-out-type NLL. This exploratory recovery localizes harmful combined predictions; the stress test exposes heterogeneous calibration failures without establishing a general structural-calibration penalty or a deployment gain over simpler alternatives.

Comment
6 pages; expanded evaluation controls and discussion, clarified uncertainty, and added references

Closest papers

Nearest neighbours by meaning across the whole index.

Partition-Matched Evaluation of Community Features under Distribution Shift in Android Malware Function-Call Graphs

cs.CR ·2026-09-21 ·Junru Zhu, Yixin Yang, Xiaoqing Ding, Ruoyu Qi
similarity 0.65
Abstract · 6 pages, 1 figure, 3 tables

Graph-based Android malware classifiers can lose accuracy under malware-type or family shifts. We test whether mesoscopic organization in function-call graphs provides shift-stable information beyond local degree profiles (LDP), global statistics, lightweight metadata, and size-matched random partitions. Using 15,000 MalNet-Tiny, Common, and Distinct graphs, six Leiden descriptors specified before evaluation, and five optimizer seeds, communities raise Tiny macro F1 from 78.7% to 81.3% but yield 29.4-point source-only Common degradation. One size-matched random partition yields 27.8-point degradation. Across five random partitions, mean degradation is 27.9 points; the 95% two-level bootstrap interval for random minus community degradation is [-4.2, 1.2] points. With metadata, the corresponding difference is -0.2 points with interval [-1.1, 0.6]. Removing modularity raises structure-only Common macro F1 from 51.9% to 53.6%. The tested signature adds IID signal but shows no repeatable shift-stability advantage, demonstrating why mesoscopic graph claims need partition-matched controls and repeated null draws.

0.40

Temporal Generalization and Explanation Stability of Control Flow Graph Neural Networks for Malware Detection

cs.CR cs.AIcs.LG ·2026-09-21 ·Md. Asif Sajeed, Md. Nazrul Islam Mondal, Md Ashraful Hossen Akash
benchmarkcode/data similarity 0.61
Abstract · 47 pages, 9 figures, 17 tables. Code available at https://github.com/Ho9pe/TG-CFG

Malware detection is a critical task in cybersecurity, and graph neural networks over control flow graphs have shown promising results for it. However, detectors are usually evaluated on a random split of a corpus collected over a single period, which cannot show how well a model generalizes to later samples. This study addresses that limitation with a strict temporal split: every model is trained on one period and scored once on a later one. Two corpora of control flow graphs, each node carrying 37 features, were extracted statically from 1,989 Windows portable executables: 459 graphs from 2024-2025 for training and 223 from 2026 for evaluation. Twelve variants and a flat-feature control were trained on the earlier corpus. The choice of message-passing operator changes robustness to the shift significantly, and every pairwise gap that survives correction separates an aggregating architecture from one built around a learned attentional readout. The ranking also reverses: the flat control, which sees node features but no topology, is the best in-distribution model and among the worst across the boundary, so a conventional benchmark would have rejected message passing. Neither recalibration nor ensembling substitutes for the operator choice. Attributions do not shift, but explanation validity is architecture-specific, and the most accurate operator on the later corpus is the hardest to explain. An architecture derived from the finding matches the best searched operator without search. The shift affects both malware and benign classes alike, so these are results about robustness to distribution shift, not malware evolution.

0.31

Residual Community Prototypes Under-Reject Held-Out Malware Families in FCG-MFD

cs.CR ·2026-09-21 ·Junru Zhu, Yixin Yang, Xiaoqing Ding, Ruoyu Qi
similarity 0.61
Abstract · 6 pages, 1 figure, 4 tables

Open-set malware-family recognition must classify known families while rejecting families absent from training. We test whether Louvain-community summaries add rejection information beyond a graph neural network embedding and dimension-matched generic topology. The study uses a deduplicated, conflict-audited FCG-MFD corpus, five held-out families, and three optimization seeds. Community features are residualized against generic topology using known-family training data before nearest-prototype scoring. Residual community does not produce stable held-out-family rejection. Ranking effects reverse across families, the false-positive rate at 95 percent unknown recall worsens for every held-out family, and a validation-fitted threshold rejects only 4.48 percent of unknown samples. Accepted-known macro F1 improves in every family, but with five independent family units the exact two-sided sign-flip p-value is 0.0625, the smallest attainable value. The score remains associated with graph scale, while simple classifier uncertainty performs better on ranking, high-recall rejection, and OSCR. In this GIN/FCG-MFD setting, community-enriched prototypes change known-class geometry without creating a stable unknown margin. Graph open-set evaluations should pair structural features with matched topology controls, operational thresholds, and held-out-family analysis.

0.35

What Malware Actually Does: An Empirical Analysis of Post-Infection Behavior Across Malware Families

cs.CR cs.AIcs.LG ·2026-11-02 ·Emmanuela Andam, Emanuel S. Grant, Naima Kaabouch
published similarity 0.60
Abstract · Accepted at 2026 Cyber Awareness and Research Symposium (CARS). This is the author's accepted manuscript

Most malware detection research focuses on identifying whether a file is malicious before it executes. Far less attention goes to what happens after infection. This paper presents an empirical analysis of post-infection malware behavior across 37 families, grounded in dynamic sandbox execution using CAPEv2. Behavioral traces are modeled as heterogeneous graphs with five node types and seven typed edge relationships, preserving relational structure that flat feature vectors discard. A voting ensemble trained on graph-derived structural features achieves 94.44% accuracy and MCC of 0.863, outperforming a flat behavioral baseline by 7.4 percentage points in MCC. A multiclass classifier distinguishes 24 malware families, and a per-family behavioral fingerprint analysis characterizes post-infection activity across registry, network, process, and API dimensions. An LLM explanation layer generates analyst-readable reports with MITRE ATT&CK mappings grounded in observed behavioral evidence. The full pipeline is deployed as HeteroScan at https://ama-andam.github.io/heteroscan.

0.38

PhantomCall: Evading ML Malware Detectors via Function Call Graph Perturbation

cs.CR ·2026-09-01 ·Md Ajwad Akil, Adrian Shuai Li, Imtiaz Karim, Arun Iyengar, Ashish Kundu +1 more
similarity 0.60
Abstract

Prior adversarial attacks on Windows PE malware detectors target raw bytes, PE headers, or intra-function control-flow graphs, leaving the function call graph (FCG) unexplored as an attack surface. Yet the FCG structure is an important feature in graph-based malware detectors. We present Phan- tomCall, a black-box attack that perturbs the FCG of Windows PE malware by injecting fully executable dummy functions at targeted call sites, adding new nodes and edges to both the CFG and FCG while preserving program semantics. We pair this structural perturbation with classifier-guided search and tunable injection parameters, effective across three archi- tecturally distinct classifiers. Evaluated on a 2025-collected Windows malware corpus against MalConv (raw-byte CNN), MalGraph (graph-based GNN), and SAFE+GNN (pure FCG GNN trained from scratch on a 2024 corpus) at two FPR thresholds, the best PhantomCall variant achieves 85-100% attack success rate across all configurations, exceeding prior state-of-the-art by up to 14.78 percentage points on MalGraph and 95.5 percentage points on SAFE+GNN, and generating evasive variants up to 2.9x faster on average across all targets. For MalConv and MalGraph, the majority of evasions require only a single call site modification, and 86-97% of evaluated evasive variants preserve the original malicious behavior in sandbox-based semantic testing across all configurations.

0.19

Behavior-Centric Malware Classification with Fine-Grained Malicious Logic Localization

cs.CR ·2026-09-29 ·Prakriti Baral, Zhuoyun Qian, Hailu Xu, Fangtian Zhong
similarity 0.56
Abstract

Effective malware analysis requires understanding not only whether a program is malicious, but also which behaviors it exhibits and where those behaviors originate in the code. Existing machine-learning-based malware detectors largely operate as black boxes, providing limited insight into the malicious logic responsible for their decisions. This paper addresses malicious behavior localization and classification at the basic-block level. We propose a behavior-centric analysis framework that decomposes malware samples into behaviors and systematically links these behaviors to their originating code regions. Using context-sensitive backward slicing from security-relevant system API calls, we reconstruct control- and data-dependency chains and represent each behavior as a structured graph of related basic blocks. A Transformer-based model captures instruction-level semantics, while a Graph Neural Network models structural dependencies within behavior graphs. The resulting representations are fused to enable accurate and interpretable classification, with attention-based attribution identifying code regions responsible for malicious behaviors. We evaluate our approach using standard classification metrics and a behavior coverage metric that measures the detection of manually labeled malicious behaviors. Our results demonstrate that the proposed framework achieves accurate malware classification while providing fine-grained, behavior-aware localization of malicious logic.

0.15

MaliciousSkillBench: A Comprehensive Benchmark for Malicious Agent Skill Detection

cs.CR cs.AI ·2026-08-20 ·Yue Wang, Yi Liu, Gelei Deng, Ying Zhang, Yuekang Li +2 more
benchmarkcode/datahighly cited similarity 0.55
Abstract · Project page: https://protectskills.github.io/MaliciousSkillBench/

Agent Skills extend LLM agents with reusable instruction packages that may also include scripts, resources, and service configuration. This creates a direct distribution channel for malicious behavior, yet existing malicious-Skill datasets are fragmented across sources, artifact formats, evidence regimes, and benign coverage; duplicated and structurally related content further complicates direct aggregation and evaluation. We present MaliciousSkillBench, a comprehensive benchmark for malicious Agent Skill detection. We consolidate 13 public sources, 11 of which contribute Core malicious artifacts, and reduce 8,414 raw malicious records to 7,539 normalized-unique identities in 4,588 operational structural families. After conservative cross-label conflict exclusion, the primary benchmark contains 9,740 Skills: 7,505 malicious and 2,235 benign. To characterize its coverage, we harmonize 11 attack categories for 4,983 malicious identities with supported source-native mappings and find substantial differences in threat composition across sources. We then evaluate three learned text detectors and three off-the-shelf Skill scanners. Learned detectors achieve 0.882-0.932 Random Macro-F1 but only 0.653-0.665 under Source-Disjoint evaluation; the strongest word TF-IDF SVM scores 0.932/0.916/0.665 on Random/structural-disjoint/Source-Disjoint while retaining 95.6% malicious recall but producing 62.4% benign FPR on held-out sources. Off-the-shelf scanners occupy different but also unsatisfactory operating regimes, reducing false positives only at the cost of sharply lower malicious recall. Together, these results show that reliable malicious-Skill detection requires both broader cross-source benchmark coverage and evaluation that jointly measures attack detection and benign over-flagging.

0.49

Beyond F1: Evaluating Coverage and Failure Recovery in AI Model Security Scanners

cs.CR cs.AI ·2026-08-27 ·Qianlong Lan, Vinothini Pandurangan, Anuj Kaul, Indranil Sanyal
benchmark similarity 0.54
Abstract

Static scanners are increasingly used to identify executable or otherwise unsafe content in machine- learning artifacts, yet conventional evaluation metrics characterize only cases where a scanner yields a usable security judgment. We evaluate ModelScan, ModelAudit, and Fickling using a controlled, artifact-backed benchmark on a synthetic corpus of 170 Pickle and PyTorch focused artifacts across 145 specimen families, 135 of which have binary security ground truth and 10 of which are intentionally malformed without labels. We explicitly distinguish non-N/A coverage, analysis completion, definitive security decisions, non-security findings, and unsupported outcomes. On labeled families, ModelAudit produced definitive security decisions for all 135 families (100%), Fickling for 110 (81.5%), and ModelScan for 67 (49.6%). Conditional on making a definitive judgment, ModelScan achieved 100% precision, recall, and F1. Fickling identified no unique true- positive families beyond those found by the combination of ModelAudit and ModelScan. Furthermore, for the 48 malicious families where ModelScan failed to complete its analysis, both ModelAudit and Fickling generated detections consistent with ground truth. These findings underscore the need to separate judgment accuracy from judgment availability, as well as incremental detection coverage from tool-level redundancy.

0.44