CS Paper Radar

Towards Tackling Application Logic Flaws through Autonomous Formal-Logic Modeling and Automated Reasoning

Yiwei Fang, Yichen Liu, Luyi Xing

cs.CR ·submitted 2026-09-09 17:58 UTC · v2

Abstract

Logic flaws pose significant challenges in the design and implementation of modern, semantically rich systems and applications, impacting security, privacy, and trust. These flaws are inherently tied to business-specific semantics and threat models, making their discovery and reasoning difficult and hard to scale. Real-world systems often exhibit diverse application features, complex protocol logic, and domain-specific threat models, necessitating substantial human effort and domain expertise for effective security analysis. In this paper, we introduce LL-Verifier, a novel, automated framework for identifying logic vulnerabilities built on (1) large language models for autonomous modeling, and (2) logic model checkers for rigorous reasoning. LL-Verifier processes natural language inputs, in particular protocol descriptions and security goals, to automatically generate formal logic models and properties expressed in a new logic language built on a generic logic language Maude, optimized for modeling arbitrary application-level semantics. These formal models are then converted into logical state machines, enabling exhaustive, rigorous verification through logic level model checking. This approach streamlines the analysis of diverse, application-level protocols deployed in real-world scenarios, offering automated, exhaustive, and precise reasoning within their logical constraints. We evaluated the high effectiveness, efficiency, and practicality of LL-Verifier by applying it to 27 access control protocols of widely used IoT devices, which come with vendor-specific logic flows and semantics. While LL-verifier tackles a hard problem in application security, i.e., automatic logic flaws discovery, our analysis uncovers a range of sophisticated logic vulnerabilities in IoT protocols and devices with serious security and privacy implications.

Comment
13 pages, 3 figures, 3 tables

Closest papers

Nearest neighbours by meaning across the whole index.

Confidence-Guided Protocol IR for LLM-Aided Security Protocol Modeling

cs.CR ·2026-09-29 ·Siqi Li, Yufan Cai, Hongshu Wang, Xinyue Zuo, Zhe Hou +1 more
code/data similarity 0.76
Abstract

Large language models offer a promising interface for translating natural-language protocol descriptions into formal security models, but their outputs remain difficult to trust without expert validation. In this paper, we present a human-in-the-loop framework for generating Tamarin-verifiable formal models of security protocols. Our key observation is that the main correctness bottleneck is the semantic accuracy rather than the syntactic validity of the intermediate protocol representation. To address this problem, we introduce a protocol intermediate representation (IR) that serves as a human-auditable semantic checkpoint between natural-language parsing and formal model generation. The IR explicitly captures protocol participants, message flows, value provenance, cryptographic operations, proof targets, and compromise assumptions. We further design an interactive interface that highlights uncertain fields and guides users to inspect the most critical semantic decisions based on model confidence before model generation. Rather than replacing formal-methods experts, our approach uses LLMs to produce auditable semantic drafts while leveraging verification tools to check the resulting formal models. Code and verification artifacts are available at https://github.com/laplace1002/TamarinAgent.git.

0.25

Discovery or Memorization? A Faithful Benchmark for Real-World Business-Logic Vulnerability Discovery

cs.CR ·2026-10-13 ·Rui Yang, Yichao Xu, Zhuohao Zhang, Jiahao Ji, Chaowei Xiao +2 more
benchmark similarity 0.73
Abstract · 22 pages, 11 figures, 4 tables

The rapid advancement of cybersecurity agents in vulnerability discovery has increased the need for benchmarks that evaluate their capabilities accurately. However, synthetic benchmarks lack realism, real-life benchmarks are prone to contamination, and many existing benchmarks have already been saturated by current models. We therefore set out to develop a benchmark that is realistic, reproducible, resistant to contamination, and challenging. Our key insight is that real-life business-logic vulnerabilities are subtle and exercise complex application interactions, making them naturally demanding. The difficulty lies in specifying evaluation criteria precise enough to faithfully measure model capabilities while keeping the tasks free of data-leakage. We introduce AnonVul, a cross-language benchmark of 132 application-level business-logic vulnerabilities, built by a semi-automated pipeline that anonymizes each task, injects oracles from expert-written specifications, and differentially verifies proof-of-concept exploits. The benchmark spans nine languages, with two evaluation tracks on black-box and grey-box settings, where each task runs in a sandbox with specialized oracles embedded in key exploitation steps. We evaluate six state-of-the-art vulnerability-discovery agents on the AnonVul benchmark, measuring discovery success, cost-effectiveness, anonymization sensitivity, oracle validity, and failure modes. Our results show that AnonVul remains challenging, with the best agent succeeding in only 23.11% of attempts. On a matched cohort of 20 vulnerabilities, restoring software identity increases discovery from 21.25% to 40.83%, highlighting the importance of controlling recognizable software knowledge when evaluating vulnerability discovery.

0.19

CVE2AP: Automated Generation of PDDL-Encoded Attack Paths via Large Language Models

cs.AI ·2026-10-02 ·Lin Cui, Vincenzo Scotti, Raffaela Mirandola
similarity 0.72
Abstract

Attack Path (AP) modeling is fundamental to cybersecurity analysis, where the Planning Domain Definition Language (PDDL) has been widely adopted to encode APs into formal and machine-verifiable representations for automated reasoning about vulnerability exploitation, attack progression, and their potential impacts. However, existing AP modeling approaches largely rely on expert-driven manual construction, limiting their scalability and ability to keep pace with rapidly evolving cyber threats. Large language models (LLMs) are promising candidates, as their extensive pre-trained knowledge and reasoning capabilities enable them to interpret and transform threat intelligence into formal representations. In this paper, we propose \textbf{CVE2AP}, an LLM-based approach for automatically generating PDDL-encoded attack paths from natural language CVE (Common Vulnerability Exposure) descriptions. CVE2AP leverages structured prompting and incorporates an error-feedback mechanism that iteratively refines the generated paths using planner-reported syntactic and solvability errors. We conduct a systematic empirical evaluation across multiple LLMs and generation configurations, assessing generation quality across syntactic, solvability and semantic dimensions, together with token consumption and generation time. The results demonstrate that CVE2AP effectively generates high-quality PDDL-encoded attack paths, achieving up to 86.9\% syntax correctness, 78.6\% solvability, and 93.1\% semantic correctness under LLM-as-expert evaluation, while \texttt{GPT-5.5} offers the best quality-cost trade-off and error feedback yields the most consistent quality improvement.

0.12

EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agents

cs.CR ·2026-09-05 ·Nanxi Li, Yingzi Ma, Yulong Cao, Chong Xiang, G. Edward Suh +4 more
benchmarkhighly cited similarity 0.67
Abstract

Large language model (LLM) agents can translate natural-language requests into real-world actions, creating safety risks from both indirect prompt injections and directly harmful requests. Existing system-level safety harnesses are typically designed by experts and applied across heterogeneous models and domains, despite substantial differences in model behavior, utility tolerance, and domain-specific safety requirements. We present EvoSafeHarness, a framework for automatically synthesizing deployment-specific safety harnesses for frozen LLM agents. Given a target model and domain specification, EvoSafeHarness jointly evolves a natural-language policy and executable enforcement logic using behavioral feedback from the target model. Each candidate is additionally evaluated through fresh-context adversarial review to filter rules that depend on benchmark artifacts. Across four agent benchmark families, EvoSafeHarness achieves a stronger safety-utility frontier than fixed expert-designed defenses. On DecodingTrust-Agent, evaluated across fifteen model-domain deployments, it reduces attack success rate from 50.9% to 12.6% for direct attacks and from 40.4% to 7.4% for indirect attacks, with a 3.3-point utility cost. On AgentDojo, it achieves 82.8% utility with 0.0% attack success rate, twice the utility of CaMeL at the same zero-attack-success operating point, while the same harness transfers unchanged to unseen AgentDyn suites. It also achieves the best score on Agent-SafetyBench for every evaluated victim model and maintains an attack success rate below 18% against an adaptive PAIR-style attacker with sixteen attempts per task. Analysis of the synthesized harnesses shows that domain semantics determine which safety relations and trajectory state must be tracked, while model and runtime behavior determine how and where those relations should be enforced.

0.38

Automating Attack Graph Construction for Agentic Pentesting. Towards Neuro-Symbolic Vulnerability Hunting

cs.CR cs.AI ·2026-09-14 ·Oliver Stevanovic, Jasmin Wachter
published similarity 0.67
Abstract · Cite as: Stevanovic, O., & Wachter, J. (2026). Automating attack graph construction for agentic pentesting: Towards neuro-symbolic vulnerabi…

Logic attack graphs grounded in scanner output provide explicit and auditable attack path reasoning LLM-based agents lack. Integrating symbolic frameworks such as MulVAL to contemporary security workflows or agentic pipelines, however, requires translating scanner evidence to initial facts, and creating domain-specific rules. We present a semi-automated pipeline that addresses this interoperability problem and depict its feasibility in a web-security case study. Our pipeline parses findings from Trivy, Semgrep, and Nmap into MulVAL predicates and uses an LLM-assisted process to construct domain-specific Datalog rules linking scanner-detectable evidence to attack techniques. MulVAL/XSB then performs symbolic inference to generate structured attack paths. We evaluate the attack-graph construction infrastructure on 54 web Capture-the-Flag tasks from CyBench within an agentic pipeline (Hybrid Reasoner); we do not evaluate the performance of the downstream agent. Every task produced at least one goal-reaching graph, and we achieve mean ground-truth vulnerability coverage of 53.7%, with 51.9% achieving full coverage; mean noise-path rate was 83.9%. With median end-to-end time of 24.9 s (MulVAL reasoning: 2.7 s) the pipeline is feasible and runtime-practical for agentic workflows, but predicate coverage, rule coverage, and path precision remain limiting factors. Next steps include semantic rule validation and agent-level comparison for graph-guided pentesting.

0.22

Correct Verdicts, Flawed Reasoning: Structured Auditing of LLM-based Vulnerability Reasoning

cs.CR cs.LGcs.SE ·2026-10-05 ·Boyue Caroline Hu, Kaivalya Ahir, Ronghao Ni, Limin Jia
benchmark similarity 0.66
Abstract

Large Language Models (LLMs) are increasingly deployed for automated software vulnerability analysis. Binary classification alone is insufficient; practitioners need explanations to triage bugs and engineer patches. Standard practice relies on Chain-of-Thought (CoT) prompting, but free-form reasoning allows models to obscure logical leaps, hallucinated execution steps, and internal inconsistencies behind plausible prose. Our manual audit reveals that approximately 60% of correct vulnerability verdicts are accompanied by fabricated or unverifiable claims, and free-form explanations allow reasoning errors to evade LLM-as-a-judge evaluation. We present Vulnerability Explanation Reasoning Auditor (VERA), an automated framework for auditing LLM vulnerability reasoning. Rather than accepting free-form text, VERA asks models to output a Structured Reasoning Record (SRR) encoding tracked pointers, memory operations, and state transitions in machine-readable fields. A multi-stage judge audits each SRR against eight reasoning failure modes using deterministic checks, with LLM calls reserved for semantic interpretation. The standardized SRR schema also enables automated mutation testing to benchmark judges at scale without human annotation. Our evaluation shows reasoning flaws occur in correct verdicts just as frequently as incorrect ones, and VERA exposes 87% of reasoning errors that free-form LLM-as-judge systematically miss.

0.42

SEMPRE: Semantic-Aware Protocol Reverse Engineering via Consistency-Driven Joint Optimization

cs.CR ·2026-10-22 ·Yihua Xu, Weijie Huang, Xiaofeng Wang, Hang Zhang, Binghui Wang +1 more
similarity 0.66
Abstract

Protocol reverse engineering (PRE) is essential for understanding undocumented or proprietary communication formats, enabling security-critical tasks such as vulnerability discovery, malware C2 analysis, and IDS signature generation. However, existing PRE techniques rely on brittle hand-written rules, under-model contextual dependencies among neighboring fields, and often fail on messages with nested sub-protocols. We present SEMPRE, a fully automated PRE framework that replaces ad-hoc heuristics with recursive LLM reasoning grounded in fine-grained dynamic program analysis. SEMPRE uses taint-flow and loop traces to guide an LLM-driven tree-of-thought search that slices messages into fields while detecting nested sub-payloads, producing a hierarchical layout. It then infers each field's type and role and iteratively refines boundaries by freezing high-confidence segments and re-running detection under semantic constraints, yielding an EM-style procedure that monotonically improves structure and semantics until convergence. Evaluated on a broad corpus of mainstream protocols against six state-of-the-art (vs. <0.4 for all baselines) and 0.72 for semantic inference (vs. 0.13 for BinPRE). Ablations confirm the contribution of each component, and a case study on a prolific real-world botnet demonstrates practical utility.

0.29

From Requirements to Attack Trees: Grounded LLM Agents for Design-Time Security Review

cs.CR cs.AI ·2026-10-02 ·Akash Iyer, Taha Demirkan, Keerthi Koneru, Aaryan Siddharthan, Sheethal Kumar +1 more
similarity 0.66
Abstract

Design-level security weaknesses can arise from requirements, trust assumptions, missing controls, and data flows before implementation begins. Existing security practices often identify these issues after code is written. We present a multi-agent LLM framework for design-time security analysis from product requirement documents and architecture diagrams. The proposed framework parses architecture diagrams into graph representations, generates misuse and failure cases, constructs attack trees, checks governance and compliance gaps, recommends mitigations, assigns enterprise security-domain tags, and produces a candidate revised architecture recommendation for expert review. The framework does not retrieve from Common Weakness Enumeration (CWE) databases at inference time. Instead, it analyzes system behavior, trust boundaries, component interactions, and data-flow assumptions. Misuse cases act as intermediate representations that link findings to system components and attack paths, while a validation and refinement loop filters unsupported findings and improves grounding, traceability, and actionability. We evaluate the framework on a Microsoft reference-labeled threat-modeling example, labeled synthetic PRD--architecture pairs, and two open-ended systems: Berty and Gas Town. The reference-labeled case supports threat-recovery and actionability analysis, while the open-ended cases evaluate validity, noise, traceability, actionability, redundancy, and attack-tree quality. Results show that architecture-informed, misuse-driven reasoning improves review quality compared with single-shot and ablation baselines. Keywords: LLM Multi-Agent Systems, Design-Time Security, Threat Modeling, Vulnerability Discovery, Architecture Diagrams, Security Analysis, Misuse Case Derivation, Attack Trees, Iterative Reasoning, Security Governance.

0.20