CS Paper Radar

Measuring Duplication, Leakage, and Rendering Dependence in Malware-as-Image Benchmarks

Stefan Larson, Kevin Leach

cs.CR ·submitted 2026-10-29 04:44 UTC · v1

Abstract

Classifying malware binaries rendered as images is a popular approach to malware family identification, and papers on the standard benchmarks for this task report accuracies of 97-99%. In this paper we audit ten of these benchmarks for near-duplicate images and test-train leakage, and retrain standard classifiers under naive and content-aware splits. Deduplicating by file hash, which is the standard practice, does not prevent leakage, because distinct binaries render to identical images. Deduplicating on image content reduces top-1 accuracy by up to 8 points and macro-F1 by up to 15 points on the released benchmarks we find leaky. The rendering step also adds duplication, since byteplots of the same binaries contain 19.7% near-duplicates where static feature vectors contain 2.2%, and changing the width at which bytes are wrapped into rows reduces accuracy by 20 to 33 points, although no byte changes. We recommend that content-level duplication and a non-image baseline be reported alongside accuracy.

Closest papers

Nearest neighbours by meaning across the whole index.

Federated Learning for Malware Image Classification under Data Heterogeneity

cs.CR cs.DCcs.LG ·2026-10-18 ·Victor Taiwo, Cemal Nisan, Muhammad Athallah, M. Emre Gursoy, Oznur Ozkasap
published similarity 0.65
Abstract

Malware image classification (MIC), in which binary files are converted into visual representations and deep neural networks are trained to identify malware families, has recently emerged as an effective approach for malware detection. Federated learning (FL) enables malware image classifiers to be trained collaboratively by multiple clients without sharing raw data. However, a fundamental challenge in FL is data heterogeneity (non-IID), which is particularly relevant but understudied in the malware context. In this paper, we address this gap via three contributions. First, we present an empirical study of federated MIC under varying degrees of data heterogeneity across three popular malware image datasets (MalImg, Virus-MNIST, and MalNet). We show that while FL achieves strong accuracy under IID conditions, both accuracy and training stability degrade as data becomes increasingly non-IID. Second, we introduce a set of round-level statistics that characterize the data contributed by participating clients in each FL round, and demonstrate that class entropy (diversity of class labels present in a given round) exhibits strong positive correlation with global model accuracy. Third, leveraging this insight, we propose an entropy-aware client selection strategy that filters candidate client subsets based on class entropy, and show that our strategy outperforms standard random selection. Our results highlight the role of client composition in federated MIC and underscore the importance of distribution-aware classifier training under data heterogeneity.

0.29

Bytes to Threats: Byte-Sequence Transformer For Malware Detection

cs.CR cs.AI ·2026-10-13 ·Abdelrahman Yassin Eljamal, Abdeltawab Hendawi
published similarity 0.65
Abstract · 7 pages, 4 figures. Accepted and published at IEEE ICAD 2026

Existing malware detection approaches heavily rely on handcrafted features derived from static or dynamic analysis of the executable files. These methods require specialized domain expertise, dedicated analysis tools, and time-consuming analysis, which limits their scalability and robustness in the face of the increasing complexity and diversity of modern malware. Consequently, there is a growing interest in approaches that operate directly on raw bytes, which contain rich semantic and structural information. However, capturing long-range dependencies within executable files remains challenging for conventional static and CNN-based methods. This paper presents MalwareFormer, a transformer-based, byte-level malware detection framework that operates directly on raw bytes, without requiring domain knowledge or feature extraction. MalwareFormer achieves an accuracy of 98.35%, a balanced accuracy of 98.32%, a macro-F1 of 98.32%, and a false positive rate of 1.94%, outperforming both baselines across all threshold-dependent classification metrics and demonstrating that transformers provide a powerful and scalable approach for byte-level malware detection.

0.21

Image-Based Techniques and Ensemble Soft Voting for Malware Classification

cs.CR ·2026-08-14 ·Sushant Rakesh Lokhande, Fabio Di Troia, Martin Jurecek, Mark Stamp
top venue similarity 0.63
Abstract · To appear as a chapter in the book "Artificial Intelligence for Cyber Defense in Emerging Threats", to be published by Springer by early 202…

In this chapter, we investigate image-based malware family classification using an ensemble learning framework and a soft voting strategy. We consider malware binaries that have been converted into images using eight distinct conversion strategies. Three complementary feature extraction tracks are applied to these images: handcrafted descriptors combining Histogram of Oriented Gradients (HOG) and Haralick texture features along with 38 statistical features; dense embeddings obtained from three pretrained neural networks (VGG16, ResNet50, and ViT-B/16), where each pretrained model is used as a frozen feature extractor with its classification head removed; and 512-dimensional embeddings derived from a custom Convolutional Neural Network (CNN) trained directly on the malware images. Each of the three feature extraction techniques is evaluated with machine learning classifiers across all eight image conversion types. The best individual results are 77.8% accuracy for the handcrafted features, 73.8% for the pretrained neural network track, and 74.8% for the custom CNN track. Then we consider various soft voting ensemble strategies, and we find that the best-performing soft voting pool--consisting of fifteen voters selected on a dedicated validation split--achieves 80.2% accuracy across the 17 malware families under consideration, a statistically significant improvement of 2.4 percentage points over the best individual model. A quantitative diversity analysis confirms that the different feature representations are complementary, with the handcrafted descriptors being the strongest contributors.

0.31

A Comparison of Malware Image Transformations Using Grad-CAM and Hybrid Learning Models

cs.CR ·2026-08-12 ·Vibha Bhavikatti, Mark Stamp
top venuebenchmark similarity 0.62
Abstract · To appear as a chapter in the book "Artificial Intelligence for Cyber Defense in Emerging Threats", to be published by Springer by early 202…

Recent studies have shown that binary-to-image representations can enable effective machine learning-based results for malware detection and classification. However, performance can vary significantly, depending on the technique used to convert binaries to images. Furthermore, the explainability and interpretability of image-based models is largely unexplored within the malware domain. In this research, we employ Gradient-weighted Class Activation Maps (Grad-CAM) as an eXplainable AI (XAI) tool, which we use to analyze eight distinct image types derived from malware samples. We provide quantitative faithfulness and stability metrics for Grad-CAM heatmaps and we compare these heatmaps to High-Resolution Class Activation Mappings (HiResCAM). We also show that Grad-CAM heatmaps can provide useful information for malware classification. Specifically, we show that a Random Forest model trained on features extracted from Grad-CAM images via a MobileNetV2 Convolutional Neural Network (CNN) model achieves a test accuracy of 0.777 across 17 malware families, exceeding a previous benchmark of 0.750 for this same dataset. A key finding of this research is that for the malware image transformations considered, accuracy and explanation faithfulness do not coincide, e.g., image transformation techniques that produce the most faithful explanations yield only mid-tier accuracy.

0.31

Neural Fingerprints for Malware Analysis: An Image-Based Metric Learning Approach with Application to Cross-Domain Classification

cs.CR ·2026-08-14 ·Manasa Deshagouni, Sayma Akther, Martin Jurecek, Mark Stamp
top venuebenchmark similarity 0.61
Abstract · To appear as a chapter in the book "Artificial Intelligence for Cyber Defense in Emerging Threats", to be published by Springer by early 202…

Identifying the family of a newly observed malware sample is a core task in threat intelligence, yet conventional classifiers must be retrained whenever a new family appears. This chapter develops an image-based metric learning approach that instead learns to extract discriminative neural fingerprints--fixed-length embeddings--from malware-as-image representations, so that family membership can be determined by nearest-neighbor search in the embedding space. The central advantage of this formulation is zero-shot capability: because the learned embedding induces a similarity metric rather than a fixed set of class boundaries, families that were never seen during training can be recognized by comparison against a gallery, with no retraining. We demonstrate this directly by training an encoder on MalNet-Images-Tiny and MalImg combined (453 families, 96,769 images) and evaluate it zero-shot on a held-out 17-family grayscale dataset with no family overlap. Using a lightweight CNN with multi-proxy anchor loss, this model attains 73.1% retrieval@1 and 90.5% open-set AUROC on families the encoder has never seen. We benchmark our embedding approach against two conventional paradigms in a same-domain setting, where all three are competitive at classifying malware into families. We further show that the learned embeddings transfer across datasets. Unlike classifiers, our embedding approach also yields interpretable similarity scores and scales to large galleries via Facebook AI Similarity Search (FAISS). Finally, we provide a comprehensive evaluation of the learned embedding space using retrieval@k, cluster purity, silhouette score, separation ratio, few-shot accuracy, and open-set detection metrics, along with robustness analysis under image perturbations.

0.31

WildFakeToolBench: Benchmarking AI-Generated Image Detectors with Web-Sourced Image Processing

cs.CV cs.CR ·2026-10-22 ·Mingjie Li, Yukun Jiang, Yicong Tan, Yang Zhang
benchmark similarity 0.61
Abstract

As AI generated images are getting popular these days, discussions on such generations are widely existed in current public forums, software repositories, and photography tutorials. Moreover, with the help of LLM-assisted code translation, malicious actors can adopt such information to evade detectors without specialized knowledge or local GPU execution of the resulting tools. To evaluate such risks, we introduce \textsc{WildFakeToolBench}, a benchmark suite of $156$ executable image-processing tools derived from $220$ public sources across $142$ domains. Each tool retains its source evidence and passes syntax, execution, and output-distortion checks without using feedback from the evaluated detectors. On $105$ generated images from five generators, we evaluate six state-of-the-art detectors with images processed by the tools in our collected bench to assess their effectiveness against such in-the-wild risks. The strongest individual tools achieve attack success rates of $25.25\%$--$100\%$ across detectors, exceeding the strongest of six selected image-processing baselines for each detector. Evaluating subsets of an eight-tool pool yields worst-case coverage above $80\%$ for every detector. Effectiveness is sparse and detector-specific: high coverage across candidates does not imply reliable one-shot evasion. These findings not only highlight the potential of such web-sourced risks, but also show the potential on establishing AI-image detector robustness evaluation framework based on those in-the-wild information.

0.23

CNN Architecture Comparison for Malware Classification with Limited Training Data

cs.CR ·2026-11-16 ·Noor Abo Hasan
similarity 0.60
Abstract

Traditional malware classification methods struggle to keep pace with rapidly evolving threats. Image-based malware classification offers a promising alternative by converting malware binaries into grayscale images for analysis with deep learning. While effective, organizations often face practical constraints in acquiring sufficient training data, especially for newer threats. This thesis investigates dual-path CNN architectures with intermediate feature fusion as a potential enhancement for malware classification, with particular focus on how varying training data size affects their comparative performance against traditional single-path approaches. This research implements and compares two models: a baseline single-path using the pre-trained VGG16 architecture and a dual-path model that combines VGG16 with a custom CNN through intermediate feature fusion. Both architectures are evaluated on a binary classification task distinguishing between two malware families (C2Lop.P trojan and Dialplatform.B worm) from the Malimg dataset. Each model is trained across four different training data proportions (20%, 40%, 60%, and 80%), with three training runs per size to ensure statistical robustness, while being evaluated on complementary test sets of varying sizes. Though both architectures achieved perfect classification performance across all training percentages, the study revealed significant differences in secondary metrics including training efficiency, robustness to noise perturbation, and architectural complexity. The findings challenge common assumptions about complex architectures, demonstrating that simpler models can offer superior efficiency and robustness without sacrificing classification performance in image-based malware analysis.

0.24

EGAMA-RC: Risk-Calibrated Evidence-Gated Adaptive Malware Analysis for Robust and Interpretable Memory-Forensic Triage

cs.CR cs.LG ·2026-08-24 ·Isaac Kofi Nti
similarity 0.58
Abstract

Machine-learning malware detectors often achieve high clean-data accuracy, but operational triage also requires evidence about uncertainty, novelty, robustness, interpretability, latency, and review cost. This paper presents EGAMA-RC, a risk-calibrated evidence-gated framework for memory-forensic malware triage. Building on SHAP-guided feature refinement, EGAMA-RC combines dataset-specific refinement, model-pool evaluation, adversarial and open-family testing, novelty scoring, explanation-conditioned evidence, and runtime-aware routing. Low-risk samples are accepted automatically, while uncertain, high-risk, or potentially novel cases are routed to review, escalation, or novelty-aware handling. Across three malware datasets and a frozen multi-seed protocol, the selected hybrid gate accepts 93.12% of pooled samples with 99.86% accepted accuracy and a 0.136% false-accept rate. Novelty calibration reduces over-restrictive review behavior while preserving a low unsafe-accept profile. XGBoost provides lightweight fast-path inference with p50/p95 latency of 0.0054/0.0059 ms per sample. The results show that dependable malware analysis requires risk-calibrated routing, novelty awareness, and controlled analyst review, not classification accuracy alone.

0.19