CS Paper Radar

Image-Based Techniques and Ensemble Soft Voting for Malware Classification

Sushant Rakesh Lokhande, Fabio Di Troia, Martin Jurecek, Mark Stamp

cs.CR ·submitted 2026-08-14 13:38 UTC · v1
top venue

Abstract

In this chapter, we investigate image-based malware family classification using an ensemble learning framework and a soft voting strategy. We consider malware binaries that have been converted into images using eight distinct conversion strategies. Three complementary feature extraction tracks are applied to these images: handcrafted descriptors combining Histogram of Oriented Gradients (HOG) and Haralick texture features along with 38 statistical features; dense embeddings obtained from three pretrained neural networks (VGG16, ResNet50, and ViT-B/16), where each pretrained model is used as a frozen feature extractor with its classification head removed; and 512-dimensional embeddings derived from a custom Convolutional Neural Network (CNN) trained directly on the malware images. Each of the three feature extraction techniques is evaluated with machine learning classifiers across all eight image conversion types. The best individual results are 77.8% accuracy for the handcrafted features, 73.8% for the pretrained neural network track, and 74.8% for the custom CNN track. Then we consider various soft voting ensemble strategies, and we find that the best-performing soft voting pool--consisting of fifteen voters selected on a dedicated validation split--achieves 80.2% accuracy across the 17 malware families under consideration, a statistically significant improvement of 2.4 percentage points over the best individual model. A quantitative diversity analysis confirms that the different feature representations are complementary, with the handcrafted descriptors being the strongest contributors.

Comment
To appear as a chapter in the book "Artificial Intelligence for Cyber Defense in Emerging Threats", to be published by Springer by early 2027

Closest papers

Nearest neighbours by meaning across the whole index.

CNN Architecture Comparison for Malware Classification with Limited Training Data

cs.CR ·2026-11-16 ·Noor Abo Hasan
similarity 0.77
Abstract

Traditional malware classification methods struggle to keep pace with rapidly evolving threats. Image-based malware classification offers a promising alternative by converting malware binaries into grayscale images for analysis with deep learning. While effective, organizations often face practical constraints in acquiring sufficient training data, especially for newer threats. This thesis investigates dual-path CNN architectures with intermediate feature fusion as a potential enhancement for malware classification, with particular focus on how varying training data size affects their comparative performance against traditional single-path approaches. This research implements and compares two models: a baseline single-path using the pre-trained VGG16 architecture and a dual-path model that combines VGG16 with a custom CNN through intermediate feature fusion. Both architectures are evaluated on a binary classification task distinguishing between two malware families (C2Lop.P trojan and Dialplatform.B worm) from the Malimg dataset. Each model is trained across four different training data proportions (20%, 40%, 60%, and 80%), with three training runs per size to ensure statistical robustness, while being evaluated on complementary test sets of varying sizes. Though both architectures achieved perfect classification performance across all training percentages, the study revealed significant differences in secondary metrics including training efficiency, robustness to noise perturbation, and architectural complexity. The findings challenge common assumptions about complex architectures, demonstrating that simpler models can offer superior efficiency and robustness without sacrificing classification performance in image-based malware analysis.

0.21

A Comparison of Malware Image Transformations Using Grad-CAM and Hybrid Learning Models

cs.CR ·2026-08-12 ·Vibha Bhavikatti, Mark Stamp
top venuebenchmark similarity 0.76
Abstract · To appear as a chapter in the book "Artificial Intelligence for Cyber Defense in Emerging Threats", to be published by Springer by early 202…

Recent studies have shown that binary-to-image representations can enable effective machine learning-based results for malware detection and classification. However, performance can vary significantly, depending on the technique used to convert binaries to images. Furthermore, the explainability and interpretability of image-based models is largely unexplored within the malware domain. In this research, we employ Gradient-weighted Class Activation Maps (Grad-CAM) as an eXplainable AI (XAI) tool, which we use to analyze eight distinct image types derived from malware samples. We provide quantitative faithfulness and stability metrics for Grad-CAM heatmaps and we compare these heatmaps to High-Resolution Class Activation Mappings (HiResCAM). We also show that Grad-CAM heatmaps can provide useful information for malware classification. Specifically, we show that a Random Forest model trained on features extracted from Grad-CAM images via a MobileNetV2 Convolutional Neural Network (CNN) model achieves a test accuracy of 0.777 across 17 malware families, exceeding a previous benchmark of 0.750 for this same dataset. A key finding of this research is that for the malware image transformations considered, accuracy and explanation faithfulness do not coincide, e.g., image transformation techniques that produce the most faithful explanations yield only mid-tier accuracy.

0.31

Neural Fingerprints for Malware Analysis: An Image-Based Metric Learning Approach with Application to Cross-Domain Classification

cs.CR ·2026-08-14 ·Manasa Deshagouni, Sayma Akther, Martin Jurecek, Mark Stamp
top venuebenchmark similarity 0.73
Abstract · To appear as a chapter in the book "Artificial Intelligence for Cyber Defense in Emerging Threats", to be published by Springer by early 202…

Identifying the family of a newly observed malware sample is a core task in threat intelligence, yet conventional classifiers must be retrained whenever a new family appears. This chapter develops an image-based metric learning approach that instead learns to extract discriminative neural fingerprints--fixed-length embeddings--from malware-as-image representations, so that family membership can be determined by nearest-neighbor search in the embedding space. The central advantage of this formulation is zero-shot capability: because the learned embedding induces a similarity metric rather than a fixed set of class boundaries, families that were never seen during training can be recognized by comparison against a gallery, with no retraining. We demonstrate this directly by training an encoder on MalNet-Images-Tiny and MalImg combined (453 families, 96,769 images) and evaluate it zero-shot on a held-out 17-family grayscale dataset with no family overlap. Using a lightweight CNN with multi-proxy anchor loss, this model attains 73.1% retrieval@1 and 90.5% open-set AUROC on families the encoder has never seen. We benchmark our embedding approach against two conventional paradigms in a same-domain setting, where all three are competitive at classifying malware into families. We further show that the learned embeddings transfer across datasets. Unlike classifiers, our embedding approach also yields interpretable similarity scores and scales to large galleries via Facebook AI Similarity Search (FAISS). Finally, we provide a comprehensive evaluation of the learned embedding space using retrieval@k, cluster purity, silhouette score, separation ratio, few-shot accuracy, and open-set detection metrics, along with robustness analysis under image perturbations.

0.31

Federated Learning for Malware Image Classification under Data Heterogeneity

cs.CR cs.DCcs.LG ·2026-10-18 ·Victor Taiwo, Cemal Nisan, Muhammad Athallah, M. Emre Gursoy, Oznur Ozkasap
published similarity 0.73
Abstract

Malware image classification (MIC), in which binary files are converted into visual representations and deep neural networks are trained to identify malware families, has recently emerged as an effective approach for malware detection. Federated learning (FL) enables malware image classifiers to be trained collaboratively by multiple clients without sharing raw data. However, a fundamental challenge in FL is data heterogeneity (non-IID), which is particularly relevant but understudied in the malware context. In this paper, we address this gap via three contributions. First, we present an empirical study of federated MIC under varying degrees of data heterogeneity across three popular malware image datasets (MalImg, Virus-MNIST, and MalNet). We show that while FL achieves strong accuracy under IID conditions, both accuracy and training stability degrade as data becomes increasingly non-IID. Second, we introduce a set of round-level statistics that characterize the data contributed by participating clients in each FL round, and demonstrate that class entropy (diversity of class labels present in a given round) exhibits strong positive correlation with global model accuracy. Third, leveraging this insight, we propose an entropy-aware client selection strategy that filters candidate client subsets based on class entropy, and show that our strategy outperforms standard random selection. Our results highlight the role of client composition in federated MIC and underscore the importance of distribution-aware classifier training under data heterogeneity.

0.29

A Hybrid Approach to Malware Detection: Integrating Few-Shot Model-Agnostic Meta-Learning with Autoencoders

cs.CR cs.AIcs.LG ·2026-10-01 ·Emmanuela Andam, Yasir Abbas Zaidi, Abdelali Hadir, Emmanuel Grant, Naima Kaabouch
published similarity 0.71
Abstract · Accepted at 2025 Cyber Awareness and Research Symposium (CARS). This is the author's accepted manuscript

Ransomware has emerged as a major cybersecurity threat, with incidents increasing in frequency and impact across critical sectors. These attacks are typically launched through phishing emails, malicious downloads, or exploitation of software vulnerabilities to gain system access. Once inside, the malware encrypts files and demands a ransom, often in cryptocurrency, for the decryption key. Conventional detection methods often struggle with novel or scarce samples, leaving systems vulnerable. To address these challenges, this paper proposes a hybrid deep learning framework that combines an Autoencoder Feature Extractor (AFE) with a Model Agnostic Meta Learning (MAML) classifier for few shot malware detection. The AFE generates compact latent features that reduce noise and dimensionality, while the MAML classifier rapidly adapts to new threats using limited labeled data. Experiments conducted on the Ransomware Dataset 2024 demonstrate the effectiveness of the framework in binary classification tasks. Across one to fifty shot settings, the proposed model consistently achieves high accuracy, F1 score, and Matthews Correlation Coefficient values, maintaining reliable classification even under extreme scarcity. These results highlight the model's robustness and effectiveness in adapting to limited data scenarios, demonstrating the potential of combining feature extraction with meta learning to enhance resilience against malware, particularly in sectors such as healthcare, manufacturing, and public infrastructure, where cyberattacks can cause significant operational and financial disruption.

0.46

Bytes to Threats: Byte-Sequence Transformer For Malware Detection

cs.CR cs.AI ·2026-10-13 ·Abdelrahman Yassin Eljamal, Abdeltawab Hendawi
published similarity 0.68
Abstract · 7 pages, 4 figures. Accepted and published at IEEE ICAD 2026

Existing malware detection approaches heavily rely on handcrafted features derived from static or dynamic analysis of the executable files. These methods require specialized domain expertise, dedicated analysis tools, and time-consuming analysis, which limits their scalability and robustness in the face of the increasing complexity and diversity of modern malware. Consequently, there is a growing interest in approaches that operate directly on raw bytes, which contain rich semantic and structural information. However, capturing long-range dependencies within executable files remains challenging for conventional static and CNN-based methods. This paper presents MalwareFormer, a transformer-based, byte-level malware detection framework that operates directly on raw bytes, without requiring domain knowledge or feature extraction. MalwareFormer achieves an accuracy of 98.35%, a balanced accuracy of 98.32%, a macro-F1 of 98.32%, and a false positive rate of 1.94%, outperforming both baselines across all threshold-dependent classification metrics and demonstrating that transformers provide a powerful and scalable approach for byte-level malware detection.

0.21

CAM-Guided Saliency Cutout and Image-Based Malware Classification

cs.CV cs.LG ·2026-08-12 ·Yasaman Ebrahimi, Martin Jurecek, Mark Stamp
top venue similarity 0.65
Abstract · To appear as a chapter in the book "Artificial Intelligence for Cyber Defense in Emerging Threats", to be published by Springer by early 202…

Dropout regularization is commonly used to reduce overfitting by removing parts of a neural network during training. For Convolutional Neural Networks (CNN), cutouts serve a somewhat analogous purpose. Cutouts can be implemented as data augmentation: the original training image is retained, and additional copies are created with regions removed. In this chapter, we test whether cutout placement can be improved by using High-Resolution Class Activation Mapping (HiResCAM). We compare four controlled training conditions: no cutout, standard random cutout, low-saliency cutout, and high-saliency cutout. We experiment using grayscale malware images from the RawMal-TF dataset (17 families with~1,000 samples per family), and for comparison to natural images, we experiment with the well-known CIFAR-100 dataset. All experiments are based on ResNet18 with~100 training epochs. For the cutout experiments, we test cutout areas of~5\%, 10\%, 20\%, and~30\%, and we consider~$M\in\{4,8}$ augmented copies per original training image. The RawMal-TF results are slightly worse for all three cutout cases (random, high and low saliency) as compared to no cutouts. In contrast, our CIFAR-100 experimental results improve slightly under low-saliency cutout. These results suggest that the value of saliency-guided cutout is domain dependent, and that malware images should not be treated as equivalent to natural images.

0.31

Malformer: A Multi-Modal Malware Detector Using Transformers

cs.CR ·2026-08-19 ·Samuel Howard, Kshitiz Aryal, Mahmoud Abdelsalam, Maanak Gupta, Andrew Wheeler +1 more
similarity 0.65
Abstract

Traditional malware detection systems that rely on a single representation of malware often fail to identify novel threats. These representations of malware binaries, also known as modalities, do not provide the models with sufficient information to discriminate among all samples. Additionally, individual representations introduce new failure modes, with some modality extraction being dependent upon the success of disassembling. Past works have integrated either additional modalities or more discriminative representations for classification. In this work, we present Malformer, a quadrimodal malware detection model that incorporates text, image, graph, and audio representations of Windows executables. We demonstrate that multimodal transformer fusion can enhance the performance of Windows malware detectors over that of unimodal and bimodal detectors. Malformer employs a combination of two RoBERTa encoders paired with a modified Vision Transformer for image data, WavLM for audio data, and an adaptive loss-weighting scheme to fuse modality-specific representations. Evaluated on a dataset of 201,549 binary samples, Malformer achieved 98.3% accuracy and an F1 score of 0.9833, outperforming both unimodal baselines and bimodal detectors by 4.6-17.6 percentage points. Malformer demonstrates that multimodal fusion provides a promising foundation for countering the growing scale of malware threats, equipping defenders with generalized and resilient detection capabilities.

0.15