CS Paper Radar

Social Engineering Attack Detection in Corporate Communications

Hanzla Hamid, Pappu Jha, Nick Rahimi

cs.LG cs.CR ·submitted 2026-11-11 01:35 UTC · v1

Abstract

Social Engineering attacks continue to pose a significant risk to the corporate world with a growing level of sophistication. It is difficult to distinguish between authentic and phishing communication via email, chat, and phone. They pose a risk of exposing trade secrets and confidential communications and can create a chain of attack from lower-level employees up to executives. Therefore, we study this growing threat using real-world datasets to build baseline machine-learning models. Our TF-IDF models classify emails as phishing or safe, achieving accuracy above 90%. We build on this by exploring whether psychological manipulation and persuasion indicators can be measured to play a significant role in detecting phishing. Instead of relying solely on surface-level lexical features, we derive interpretable lexicon-based indicators of urgency, trust, greed, and other persuasion cues from the combined email dataset. By evaluating these manipulation-oriented features on their own, we measure how much signal they carry (ROC-AUC of 0.74 on a held-out test split) and test whether they transfer to held-out email sources.

Closest papers

Nearest neighbours by meaning across the whole index.

Nebulon Enterprise Simulated Threats for Phishing Research (NEST-Phish): A Synthetic Enterprise Phishing Email Dataset for Behavioral and Machine-Learning Research

cs.CR ·2026-09-03 ·Emily J. Winokur, Lauren S. Treiman, Allen G. Moore, Paul Schutte, Joe Ingram +1 more
benchmarkdatasetcode/data similarity 0.79
Abstract · Dataset paper; associated public dataset available on Kaggle: https://www.kaggle.com/datasets/emilywinokur/synthetic-email-corpus

Phishing remains one of the most persistent cyber threats, yet publicly shareable datasets for studying phishing in realistic enterprise email settings remain limited. To address this gap, we introduce a synthetic enterprise phishing email dataset built around a fictitious organization, Nebulon. The dataset spans a broad set of workplace communication themes and includes matched synthetic legitimate and phishing emails with interpretable phishing-cue annotations. Here, ``legitimate'' denotes the non-phishing class, not legitimately occurring organizational emails. Human-subject categorizations and classifier evaluations show that the dataset supports meaningful variation in phishing judgments while also providing learnable signal for supervised detection. This publicly released resource is intended to support future work on phishing detection, human susceptibility, explainability, and benchmark development in enterprise-like contexts.

0.21

A Large-Scale Empirical Study of Modern Phishing Email Content

cs.CR ·2026-09-25 ·Jaehwan Park, Woonghee Lee, Fujiao Ji, Doowon Kim
similarity 0.72
Abstract · 22 pages

Phishing remains one of the most pervasive threats to Internet users, and email remains its predominant delivery channel. Email content is the attack surface of phishing: it is what the victim reads and what automated defenses inspect. Yet the composition of modern phishing content is poorly measured. Prior work has characterized dimensions such as theme, call-to-action (CTA), and impersonation, but not at scale, and their associations and temporal changes remain unclear, owing to small or source-specific corpora, bag-of-words topic models, and a focus on text alone. We present a content-focused measurement study of 2.9M distinct real-world phishing emails collected over 13 months (June 2025 - June 2026) in collaboration with the Anti-Phishing Working Group (APWG). We treat each email as a composite artifact comprising message text and its attachments: 272K images, 143K PDFs, and 57K calendar invitations. Using an LLM pipeline validated against human-annotated samples, we analyze these components along three dimensions (theme, CTA, and impersonation), examine the associations among them, and measure longer-term change against a historical dataset. We find that attackers diversify what they use to deceive but converge on how victims should respond: no theme exceeds 21.3% of emails, while a single CTA, URL navigation, accounts for 73.0%. CTA and impersonation choices are conditioned on theme. Attachments play three roles: images supplement the message text, PDFs substitute for it by carrying the pretext, and calendar invitations reinforce it by replicating interaction endpoints into a persistent medium. Over the longer term, the dominant CTA for invoice-themed phishing shifted from URL navigation to offline communication, rising from 6.7% in 2015 to 46.9% in 2025.

0.26

Engineered Persuasion: Evaluating Personalized Pretexts in LLM-Generated Spear Phishing

cs.CR ·2026-09-03 ·Jerson Francia, Derek Hansen, Benjamin Schooley, Shydra Valynn Murray
similarity 0.70
Abstract

Large language models can insert workplace details into phishing pretexts at low cost, but those details may either support or undermine a message's credibility. We recruited 180 U.S. working adults to evaluate simulated, AI-generated phishing emails in a disclosed survey. The emails used four cumulative levels of information: workplace (Level 1); recipient name and job title; job responsibilities; and coworker/shared-project context (Level 4). Participants rated each message's convincingness from 0 to 100, chose one stated action (open the link, investigate, delete, or report), and explained why their highest- and lowest-rated messages stood out. Across 1,436 valid evaluations, convincingness increased by 2.40 points per personalization level in a sensitivity analysis, while the odds of expressing click intention increased by 28\% per level. Among participants who did not express an intention to click, investigation remained common, reporting declined, and deletion increased. A post-hoc descriptive analysis found higher ratings and click intention for messages from a named person who referenced a supplied coworker than for messages from a department or entity. Qualitative coding showed why added detail could help or hurt: details that matched participants' roles and routines supported credibility, while incorrect, vague, or channel-inappropriate details raised suspicion. Together, the results highlight that personalization is not simply a matter of adding more details: it depends on whether the pretext fits the recipient's work context. We discuss how this distinction can inform workplace cybersecurity training.

0.10

Breaking and Defending LLM-Powered Social Media Bot Detection Systems

cs.AI ·2026-08-16 ·Nof Orenstein, Yoni Birman
published similarity 0.68
Abstract · Accepted at ACISP 2026 (Australasian Conference on Information Security and Privacy). Also accepted as a poster at IEEE Symposium on Securit…

The rise of social media bots poses a persistent threat, enabling misinformation, opinion manipulation, and the erosion of trust in online platforms. To combat this, machine learning systems have been developed to detect and limit bot activity, but attackers continuously adapt through techniques such as adversarial learning and behavior imitation, fueling an ongoing arms race between bots and detection tools. Recent advances in large language models (LLMs) have significantly improved bot detection by enabling deeper semantic and contextual analysis of accounts and their content. However, this shift also introduces new attack surfaces, allowing adversaries to craft exploits that directly target the reasoning and generation mechanisms of LLM-based classifiers. Industry tools such as Anthropic's Claude Code Security similarly leverage LLMs for security-critical decisions, further motivating a careful study of their attack surfaces. In this work, we investigate both the offensive and defensive aspects of LLM-powered, threat-specific cybersecurity applications. While centered on the challenge of social media bot detection, our methodology and insights generalize to a broad class of LLM-powered cybersecurity systems, including phishing detection, email classification, and fraud analysis. We introduce two novel adversarial attack strategies that systematically exploit the semantic and contextual weaknesses of LLM-based classifiers, degrading their detection accuracy by up to 48%. To counter these threats, we propose a robust multi-LLM defense architecture designed to preserve detection reliability under adaptive adversarial conditions. Our solution, LSABRE (LLM-powered Social Adversarial Bot Recognition Ensemble), is a multi-LLM framework that substantially improves robustness across a range of attacks, maintaining 86% detection accuracy even under strong, adaptive adversarial pressure.

0.40

Vishing-Tactics-Bench: Forecasting Exploitation Trajectories in Voice Phishing Calls

cs.CL ·2026-09-07 ·Jeongmin Lee, Dongmyung Sul, Seung Yun, Jinxia Huang
top venuebenchmark similarity 0.68
Abstract · 16 pages, 3 figures. Accepted to Findings of EMNLP 2026

Voice phishing (vishing) unfolds in real time; by the time a call has ended and post-hoc classification is possible, the harm has already been done. The more actionable question is which concrete harm (Information Gathering or Financial Exploitation) an ongoing call is tactically progressing toward. We present Vishing-Tactics-Bench, a benchmark grounded in Endsley's situation-awareness (SA) framework that recasts vishing defense from after-the-fact fraud classification to harm projection: predicting at each turn whether the call will reach either terminal harm. We adapt MITRE ATT&CK to vishing as a 6-tactic taxonomy (Vishing-Tactics) and label 35,340 scammer utterances across 5,645 synthetic Chinese calls. We define Exploitation Trajectory Forecasting, a survival-style protocol over the two terminal harms with three metrics: AP@k, C-index, and divergence error. Baselines ranging from a Markov heuristic to fine-tuned LLMs show that the tactical trajectory serves as an interpretable representation of the call's tactical state, supporting harm-specific forecasting, which can then be used for the downstream application of intervention selection; a stratified lead-time analysis at a tight false-alarm budget further identifies at what point in a call the trajectory signal yields early warning.

0.66

Reddit Community Signals as a Supplementary Domain-Reputation Feature in Spam Classification: A Conservative Controlled Evaluation

cs.CR ·2026-11-01 ·Fatima Salcedo, Jide Edu
similarity 0.67
Abstract · 25 pages

Email spam and phishing remain persistent threats, and purely algorithmic classifiers may struggle with unfamiliar senders, evolving message content, and cold-start domains. This study evaluates whether Reddit-derived community signals can serve as a supplementary domain-reputation feature in email spam classification. We use a preprocessed SpamAssassin-derived corpus of 5,582 emails and compare three feature configurations: (1) a baseline using TF-IDF and structural features; (2) a traditional-enhanced configuration adding urgency and action scores; and (3) a Reddit-enhanced configuration adding a binary discussion-presence indicator, a keyword-based Reddit score, and an urgency-Reddit interaction term. We train Logistic Regression, Random Forest, and Support Vector Machine models. In the preliminary single-split evaluation, the Random Forest model with Reddit-enhanced features achieves 97.94% accuracy, a 0.55 percentage-point improvement over the baseline, although the improvement is not statistically significant in the primary test split. Logistic Regression performance declines when additional features are introduced, and the SVM performs poorly and is treated as a negative control. Reddit signals appear most informative for rare domains, but approximately 70% of emails receive no Reddit-derived signal. Because the Reddit data are temporally later than the email corpus, we do not interpret the results as evidence of real-time detection. Instead, the study provides a conservative evaluation of community-derived signals as a supplementary, asynchronous domain-reputation feature and identifies the additional validation required before production deployment, including domain-grouped validation, temporally aligned contemporary data, and adversarial robustness testing.

0.27

AgentPhishing: A Benchmark of LLM Email Agents Under Adaptive Attack

cs.CR cs.AI ·2026-10-22 ·Justin Tan, Miao Li, Ding Zhao
benchmark similarity 0.67
Abstract · 28 pages, 6 figures

Computer-use agents increasingly handle private information and act on users' behalf across desktop and web applications, including email. Phishing therefore poses privacy and security risks that depend not only on whether agents recognize malicious messages, but also on how they respond. Existing phishing datasets were developed primarily for conventional detection, leaving agent susceptibility to persuasive malicious requests insufficiently characterized. To expose vulnerabilities that pre-existing datasets may miss, we introduce AgentPhishing, a benchmark for evaluating whether persuasive phishing attacks can evade detection and induce LLM email agents to comply with malicious requests. We develop PhishOpt, which optimizes reusable email-generation skills against individual agents under fixed benchmark constraints and a realism constraint. We analyze the generated emails using Cialdini's persuasion principles to identify strategies associated with agent failures and use these findings to construct a universal generation skill. Our evaluation shows that agents can handle pre-existing phishing emails safely yet remain susceptible to attacks adapted to agent behavior, while in an AgentPhishing evaluation of nine agents, four exhibit phishing-detection rates below $10\%$ and malicious-request compliance rates above $45\%$. These results highlight the limits of inferring agent safety from conventional phishing evaluations and the need to assess how agents respond to persuasive malicious requests.

0.26

PHISHREV+: An ASP-Based Post-Hoc Defeasible Machine Learning Framework for Phishing Detection

cs.AI cs.LG ·2026-11-06 ·Mainak Sen, Kumar Sanka Ray, Amlan Chakrabarti
benchmark similarity 0.66
Abstract

Phishing website detection remains a critical cybersecurity challenge. Although machine learning classifiers achieve high accuracy on benchmark datasets, they suffer from a fundamental limitation: once trained, their decision boundaries are fixed and cannot incorporate new contextual evidence without retraining the model. This work presents \textbf{PHISHREV+}, a hybrid framework that instantiates the notion of \textit{post-hoc defeasible machine learning}. This framework does not replace the classifier. It treats the initial classifier predictions as defeasible beliefs and revises them through an Answer Set Programming based non-monotonic reasoning(NMR) layer. The framework operates in two phases. In the machine learning phase, four classifiers (SVM, KNN, DT and RF) are trained on the 87 feature benchmark dataset. In the post-hoc NMR phase, four defeasible rules derived from exploratory data analysis of the training set, and validated through Chi-square and Mann-Whitney U tests, formally revise initial predictions. The proposed framework achieves consistent false positive reduction across all our classifiers SVM (41$\rightarrow$39), KNN (47$\rightarrow$46), DT (62$\rightarrow$61), and RF (52$\rightarrow$51) while maintaining complete false negative parity with zero increase overhead. A comprehensive ablation studies across seven rule configurations confirms that the hierarchical combination of all four defeasible rules is necessary to achieve this optimal balance. Instance level explainability is provided through eXplainable ASP (XASP), which generates formal derivation graphs showing which defeasible rules fired and why the initial belief was retracted, a capability absent from all existing phishing detection approaches.

0.13