CS Paper Radar

Black-Box Adversarial Example Attacks on Machine Learning based Election Systems

Subek Acharya, Syed Ayon, Khaleque Md Aashiq Kamal, Alina Barnett, Surya Eada, Kaleel Mahmood

cs.CR cs.AI ·submitted 2026-11-01 20:06 UTC · v1

Abstract

Machine learning models have shown exceptional accuracy for ballot classification in election systems. Recent research has demonstrated that these ballot classification models are vulnerable to adversarial example attacks. This is a critical security issue because a successful attack could alter the outcome of an election and undermine the democratic process. However, all adversarial example attacks in the voting domain have assumed a white-box threat model, where the classifier parameters are known. In this work, we advance the field of machine learning in election systems by analyzing a realistic black-box adversary. We implement and evaluate eleven state-of-the-art black-box attacks: query based attacks ADBA, Square, SurFree, RayS, and SA-MOO. We also test adaptive transfer based black-box attacks with APGD-$l_\infty$, APGD-$l_2$, APGD-$l_1$, PGD $l_0$, PGD $l_0$+$l_\infty$, and PGD $l_0$+$σ$. Our attacks span multiple different norms ($l_\infty$, $l_2$, $l_1$, and $l_0$) and three different black-box adversarial threat models (decision based, score based and transfer based). In addition to digital analyses, we physically print and scan $20,000$ adversarial examples to evaluate the real-world performance of different black-box attacks. Our finding demonstrate an even larger physical-digital robustness gap exists for black-box adversarial examples than for white-box adversarial examples. Lastly, we conduct the first ever adversarial example transferability study in the voting domain and test adversarial training and barrier zones as two possible black-box defense strategies.

Closest papers

Nearest neighbours by meaning across the whole index.

Adversarial Robustness of Kernel Machines: A Comprehensive Analysis

cs.LG cs.AI ·2026-11-05 ·Rahul Kumar Yadav, Tunir Ghosh, Sudhendu Ahir, Parthe Pandit, Arjun Bhagoji
similarity 0.71
Abstract

Kernel machines are an important class of models in machine learning. However, the question of how robust modern kernel machines are to adversarial examples has not been studied comprehensively. In this paper, we provide a thorough empirical investigation of the adversarial robustness of kernel-based models for both binary and multiclass classification tasks, via white-box and black-box evasion attacks. We evaluate the native robustness of kernel machines, random feature classifiers, and the relatively newer Recursive Feature Machines. In addition, we provide an algorithm for approximate adversarial training of kernel machines. Our experiments on classification tasks on image and tabular datasets strongly indicate that kernel machines are natively more robust than neural networks, i.e., their performance degrades more gracefully against adversarial perturbations.

0.13

Mitigating Black-Box Score-Based Query Attacks via Gradient Confusion

cs.LG cs.AIcs.CR ·2026-10-12 ·Ali Javeri, Javad Hezareh, Radin Shahdaei, AmirMahdi Sadeghzadeh
similarity 0.67
Abstract

Score-based black-box adversarial attacks pose a serious threat to deployed machine learning systems by exploiting repeated model queries to craft adversarial examples, requiring neither access to model parameters nor architecture. Existing defenses predominantly rely on stochastic output perturbations, which fail to prevent determined attackers from recovering stable gradient estimates by increasing query budgets or averaging over randomness. In this work, we propose the Gradient Confusion Defense (GCD), a retraining-free, deterministic test-time wrapper that actively shapes the probability vector returned to querying adversaries. By solving a constrained optimization problem grounded in the model's Jacobian, GCD aligns the exposed logit-gradient with a target confusion direction that is maximally misaligned with the true adversarial loss gradient---increasing the query complexity of attacks without sacrificing predictive utility or top-1 accuracy. We provide rigorous theoretical analysis demonstrating that GCD provably slows gradient-estimation-based attacks and induces systematic accept/reject errors in random search-based attacks, with the confusion effect amplifying as the fraction of defended queries increases. Extensive experiments on CIFAR-10 and ImageNet across multiple architectures (ResNet-18, ResNet-50, ViT-B/16) and four state-of-the-art attacks confirm these predictions: GCD achieves state-of-the-art robustness, while incurring zero clean accuracy degradation. %Crucially, unlike randomization-based defenses, GCD's deterministic structure provides resilience against adaptive attacks that exploit stochasticity, making it well-suited for security-critical deployment settings.

0.12

Everyone's a little bit racist: a novel ML backdoor attack in black-box mode

cs.CR ·2026-10-15 ·Michele Cerreta, Gabriele Costa
similarity 0.67
Abstract

Backdoors are among the most impactful attacks against Machine Learning (ML), yet most existing methods assume some form of training-time poisoning or direct model manipulation. This is often considered sufficient to assume that existing systems are not exposed to actual risks. In this paper, we introduce Backdoor-as-a-Service, a model-agnostic, fully black-box technique that, instead of installing a backdoor, activates it in the form of a latent secondary task leveraging on an already present behavior. Our tool uses genetic search to discover input transformations that induce systematic output differences aligned with an attacker-defined task, without modifying training data, architecture, or weights. We verify our approach against two targets. First, we target a MNIST digits classifier that we coerce into implementing a binary letter-recognition task. Then, we attack a production-grade NSFW classifier using face-specific perturbations to turn it into a proxy for demographic-group classification. In both cases, our method successfully enables the target backdoor. Finally, we discuss the existing countermeasures to assess the actual risk for the deployed systems.

0.15

ATLAS-AL: Adaptive Trust-Region for Latent Adversarial Searches via Active Learning

cs.LG cs.CRcs.CV ·2026-10-05 ·Marsalis Gibson, Claire Tomlin, Shankar Sastry
similarity 0.66
Abstract · 9 pages main body, plus 10 additional pages for references and appendix

Security evaluation of learning-based systems requires more than just testing the system against a fixed collection of attacks. It requires adaptive mechanisms that can efficiently discover \textit{sets} of inputs that induce model failure. We introduce ATLAS (Adaptive Trust-Regions for Latent Adversarial Searches), which is a query-based framework that discovers adversarial input sets for black-box learning systems. ATLAS casts attack generation as an active learning level set estimation problem then combines calibrated approximations with a local-global sampling architecture to find regions of the input space that contain adversarial examples. Once discovered, ATLAS is designed to sample points within these adversarial regions to build adversarial sets that accurately represent the state of robustness of the target model. When applied on toy experiments, we find that ATLAS is able to recover more of the adversarial region under a limited query budget than does previous work. When applied to standard and adversarially trained MNIST, CIFAR, and ImageNet model targets, ATLAS produces better representative attacks than other query-based black-box attacks (NES, SignHunter, BayesOpt). ATLAS represents an automated red-teaming framework that can be used for both analyzing the robustness of learning-based systems under development and continuous auditing to see how the robustness of a system changes over time.

0.26

Reinforcement Learning Inspired Black-box Adversarial Attacks for Computer Vision

cs.LG cs.CRcs.CV ·2026-09-21 ·Florian Krone, Elena Hoemann, Sven Hallerbach
similarity 0.65
Abstract

Neural networks, both convolution or transformer based, are essential for modern computer vision systems. However, they are vulnerable to small perturbations, almost imperceptible to humans, which significantly alter the model's prediction. These adversarial attacks are often considered to be a significant threat to the implementation of neural networks in safety-critical applications. Most attacks utilize the white-box threat model and therefore require full access to the target model, making them unrealistic to use in practice. We propose a novel approach under the more realistic black-box threat model that utilizes concepts from reinforcement learning to optimize perturbations with a non-differentiable target model. Reinforcement learning algorithms have already been optimized to be query efficient, making them an ideal starting point when designing black-box adversarial attacks. We show the success of our reinforcement learning inspired black-box adversarial attack (RIBA) in generating adversarial perturbations using only a small number of queries to the target model, by comparing it to state of the art attacks on different models on the Cifar10 and ImageNet data sets. RIBA takes $25.4\%$ fewer median queries to generate attacked images against a ResNet-18 on Cifar10 and $22.5\%$ fewer median queries to fool a Vit-B/16 model on ImageNet. Additionally, we demonstrate that RIBA can match the performance of white-box attacks on an adversarially trained model.

0.13

JevAdvBench: A Benchmark and Black-Box Attacks for Reinforcement Learning for Calibrated Decisions Models

cs.CR cs.AIcs.CL ·2026-09-25 ·Jianyi Hu, Hangtao Zhang, Yi Liu, Yeqi Zeng, Li Zeng +3 more
benchmarkcode/datahighly cited similarity 0.65
Abstract · 12 pages, 5 figures. Project page: https://JevAdvBench.github.io/JevAdvBench/

Typed decision models (TDMs), such as Jev, answer a question about an input with a probability, an option, or an ordinal level. Web services execute these decisions automatically above a confidence threshold. An attacker who writes part of the input can change a decision that the service then executes. Current adversarial benchmarks cannot measure this risk, because they judge generated text. A TDM writes no text, and its wrong decisions have a valid format. We present JevAdvBench, to our knowledge the first unified benchmark for the adversarial robustness of TDMs. It contains 862 questions with audited answers and twelve black-box attack methods that edit the input or the options by adaptive search. Because a changed decision is not always wrong, a review keeps only 5,586 answer-preserving variants, and we attack only correct decisions. The attacks flip 46.3% of Jev's correct decisions, while identical re-runs flip only 0.17%. A flawed argument without a command flips 77.2%, close to direct prompt injection at 92.0%. The same variants flip 34.0% to 51.7% of correct decisions on eight open TDMs. At a 0.8 threshold, 23.5% of the decisions that the service still executes are wrong. On Agentgateway, the Jev guardrail rejects a jailbreak request but accepts it with an inserted argument. Web services should therefore treat TDM inputs as untrusted and should not act on confidence alone. Project website: https://JevAdvBench.github.io/JevAdvBench/

0.65

FlipJev: Fast Black-Box Red Teaming Finds Universal Adversarial Suffixes for Typed Decision Models

cs.CR cs.AI ·2026-10-28 ·Xiaogeng Liu, Xinyan Wang, Chaowei Xiao
code/data similarity 0.65
Abstract · Pre-print. Code: https://github.com/SaFo-Lab/FlipJev, Webpage: https://flipjev.github.io

Typed decision models are an emerging class of language models that make decisions instead of generating free-text responses. They answer in a single fast and cheap query and are increasingly used, including in security-sensitive tasks. However, red teaming of these models leaves two gaps: existing adaptive attacks do not use their speed, and few studies have searched for universal adversarial examples. We fill both gaps on Jev, the representative typed decision model, with a quick black-box query-based attack that calls no attacker language model. We conduct extensive experiments on 12 datasets with different data types and application scenarios. The results show that Jev resists common prompt injections: no strategy changes a correct answer in 10,240 paired tests. In contrast, our attack finds short suffixes that flip confident answers within minutes. These suffixes are universal: one suffix per dataset flips 43% of unseen documents. Because typed decision models, unlike classic NLP models, can serve many tasks, we test whether these suffixes transfer across tasks. We find that one suffix generated from one task can flip 90-100% of the samples in other tasks. We therefore believe the evaluation of typed decision models must include strict adaptive attacks to fully reveal their robustness.

0.25

Evasion Attacks on Cost-Utility-Based Adversarial Training for Online AutoML in IoT Networks

cs.CR cs.LG ·2026-09-25 ·Chukwunonso Henry Nwokoye, Wajiha Zaheer, Khalil El-Khatib, Li Yang
publishedcode/data similarity 0.65
Abstract · Accepted to the IEEE Middle East Conference on Communications and Networking (MECOM 2026). Code is available at: https://github.com/ChiNonso…

As Internet of Things (IoT) networks increasingly depend on machine learning for anomaly, malware, intrusion detection, and network monitoring, such systems have become attractive targets for evasion attacks. Evasion attacks pose a major security risk because an adversary intentionally modifies input data to mislead a trained model into producing incorrect predictions while evading detection. This study evaluates the impact of black-box evasion attacks on a cost-utility-based adversarial training defense strategy in an Online AutoML context for IoT networks. Specifically, evasion attacks were applied to online learners, including Hoeffding Tree (HT), Leveraging Bagging (LB), Streaming Random Patches (SRP), Hoeffding Adaptive Tree (HAT), and Adaptive Random Forest (ARF). By developing naive and adversarially trained (AT) versions of these online learners, we generated clean and adversarial accuracies for each model. The results show that the AT versions of LB and SRP performed best, achieving the highest adversarial accuracy (0.985) and high clean accuracy (0.993) at the highest cost budget of 1.00, with a maximum accuracy reduction of only 0.8%. Finally, drift detection was conducted using the Early Drift Detection Method (EDDM).

0.42