CS Paper Radar

Developing Security Metrics for Automated Compliance Checking

Immanuel Kunz, Nico Haas, Angelika Schneider, Christian Banse

cs.CR ·submitted 2026-10-23 14:20 UTC · v1
publishedcode/data

Abstract

To automatically check for compliance with security-related standards, prose requirements must be translated to technical properties to make them measurable. A multitude of metrics have been proposed in past literature, but they are lacking a systematic methodology for their elicitation. Without such a methodology, metrics remain inconsistent in their data models and abstraction levels, they exhibit considerable overlaps and are difficult to combine to comprehensively cover a standard's requirement. In this paper, we develop a data model for security metrics based on a systematization of compliance requirements, and we propose a methodology for their elicitation. The method is based on an ontology that enables a systematic development of metrics with a comparable degree of abstraction. We also present our initial work on an open-source effort that demonstrates that our approach also supports a collaborative development of metrics.

Comment
15 pages, 1 figure, 1 table. Author-accepted manuscript of a paper published in the proceedings of IFIP SEC 2026. The final authenticated version is available from Springer Nature at https://doi.org/10.1007/978-3-032-27993-4_20
Journal ref
In: Proceedings of IFIP SEC 2026, IFIP Advances in Information and Communication Technology, Springer, 2026
DOI
10.1007/978-3-032-27993-4_20

Closest papers

Nearest neighbours by meaning across the whole index.

An Empirical Study and Assessment of EU AI Act Compliance Checkers

cs.AI cs.CR ·2026-09-28 ·Zhen Tao, Alize Kahraman, Shidong Pan, Zhenchang Xing, Chiara Ullstein +2 more
similarity 0.66
Abstract

The EU AI Act introduces extensive compliance requirements for organizations that develop, deploy, or integrate AI systems. Many of these requirements are directly relevant to security and privacy, while also addressing closely related issues such as data governance, transparency, accuracy, and robustness. However, stakeholders such as small-to-medium businesses and individual developers often lack the legal expertise required to interpret these obligations and translate them into engineering and governance practices. This disconnect creates challenges for implementing the EU AI Act and may lead to missing safeguards or misdirected development and deployment efforts. To address this, various automated EU AI Act compliance checkers (AIACCs) have emerged, claiming to streamline compliance assessments and provide practical guidance. In this paper, we present the first empirical study and assessment of AIACCs. We characterize 12 mainstream AIACCs across multiple dimensions, evaluate their legal coverage and alignment, and analyze checker-generated compliance reports for structure, determinacy, and actionability. We find that the quality of AIACCs varies significantly and that they currently can only serve as early-stage orientation tools. Specifically, we observe inconsistent interaction modes and user-friendliness, a tendency to overly simplify or omit key obligations, and a failure to provide determinate, actionable guidance. As a result, reliance on the current generation of AIACCs may foster a false sense of compliance. With our study, we provide a critical baseline of the current AIACC landscape. We further offer design principles for the implementation of more reliable compliance-support tools.

0.25

What Must Be Done Here? Using a Bounded Application-Security Ontology at the Point of Change

cs.CR cs.SE ·2026-10-31 ·Pedro Farinha
similarity 0.64
Abstract · 8 pages, 2 figures. Accepted at Semantic Shields 3: the 3rd International Workshop on Modeling for Cybersecurity, co-located with JOWO 2026 …

The security guidance that bears on code is real but scattered. It lives across secure-development frameworks, verification standards, controls catalogues, threat taxonomies, and, increasingly, normative regimes such as the EU's NIS2 -- and most of it is written for readers other than the engineer, or the AI assistant, at a point of change. Existing cybersecurity ontologies largely model attacks, indicators, and risk for detection and analysis, not the engineering-practice substance one must act on. This paper presents a use case for an existing bounded application-security ontology, AppSec Core, whose construction and validation are reported separately. We show how it is used. It makes the engineering-actionable fraction of those authorities addressable as a typed working set: the control objectives that apply, the practices that realize them, the mechanisms that implement them, and the evidence artifacts they expect, each traceable to the sources that justify it. We reach that working set from two entry points that recur in practice. Forward, from a code change: a change to authentication middleware scopes to its governing domain slice and returns the obligations the change must satisfy. Inverse, from a normative obligation: "make this NIS2-compliant" resolves a generic legal duty into the concrete engineering obligations it implies across slices, surfacing precisely the fraction of a mostly-governance regulation that lands in code. The same working set is what a human engineer reads and what an LLM-grounded assistant is handed as grounding. The use turns "read every applicable authority" into "satisfy this typed working set", for a code change or a compliance duty alike. The contribution is the demonstrated use; the artefact, its validation, and its governance are companion work.

0.65

AspisAI: A Canonical, Machine-Interpretable Governance Framework for Automated Multi-Standard Compliance Monitoring

cs.CR cs.CYcs.SE ·2026-09-09 ·Tsafac Nkombong Regine Cyrille, Hasan Dag, Reiner Creutzburg, Knut Haufe
publishedcode/data similarity 0.63
Abstract · 6 pages, 2 figures. Accepted at IACyC 2026, the 2nd CyberMACS International Applied Cybersecurity Conference

Organisations operating in regulated and critical-infrastructure sectors must satisfy multiple, heterogeneous cybersecurity and privacy instruments simultaneously, including but not limited to ISO/IEC~27001, the NIST Cybersecurity Framework~2.0, Cyber Essentials, and the GDPR. In practice, these obligations are managed through manual mappings, spreadsheet-based tracking, and periodic audits that are costly to maintain, inconsistent across standards, and weak in traceability. This paper presents \emph{AspisAI}, a bounded, standard-agnostic governance framework that translates selected requirements from several frameworks into a canonical, machine-interpretable control model, and evaluates submitted evidence against condition-based decision rules to produce explainable, traceable compliance determinations. Within a bounded scope of 26 representative requirements, the framework is evaluated in a controlled simulation against five governance-oriented criteria and, critically, against two external reference points that mitigate the circularity of single-author evaluation: its cross-standard mappings are validated against NIST's own published informative references, with 57\,\% exact agreement and divergences confined to same-family controls, and the framework is applied to real third-party evidence from the OpenSSF Scorecard, surfacing genuine governance gaps in a live open-source project. The controlled results, comprising full requirement encoding, 88.5\,\% mapping coverage, complete traceability, and correct detection of all introduced gaps, establish functional correctness, while the external validation provides evidence of applicability beyond the simulation. The contribution is therefore a demonstration that a canonical, provenance-preserving governance model can render multi-standard compliance both automatable and auditable.

0.49

A Hybrid LLM-Based Framework for Automated Security Annotation Generation in Business Process Models

cs.CR cs.AIcs.SE ·2026-08-14 ·Md Kamrul Islam, Tiphaine Henry, Mattia Salnitri, Julius Köpke, Sami Souihi
top venue similarity 0.61
Abstract

The modelling and analysis of secure business processes require the incorporation of security annotations into process models. Although BPMN extensions, including SecBPMN2, exist for this purpose, the derivation of accurate and complete security annotations from natural-language specifications remains a manual, expert-intensive, and error-prone task. This paper presents a hybrid framework that takes a BPMN process model and a security requirements document as input and automatically generates security annotations adhering to the SecBPMN2 specification. The approach combines Large Language Model (LLM)--based semantic extraction with schema-constrained mapping, rule-based normalization, and deterministic validation. The framework is evaluated comprehensively on a curated dataset of 27 process models from various domains. The results indicate that it consistently produces structurally valid SecBPMN2 annotations with high schema completeness. Compared to human security analysts, the system achieves substantially higher precision (0.58 vs. 0.29) while maintaining comparable recall (0.52 vs. 0.50) and reduces erroneous or misplaced annotations by nearly 50%. In addition, annotation generation is significantly faster than manual annotation. These findings demonstrate that hybrid LLM- and rule-based automation can reduce modeling effort while improving consistency and reliability, thereby providing a scalable foundation for security-by-design BPM.

0.35

ARCCS: An Automated Regulatory Compliance Checking System

cs.CL ·2026-10-01 ·Giorgos Filandrianos, José Menezes, Chrysoula Zerva, Alessandro Gianola
top venuebenchmarkcode/data similarity 0.61
Abstract · This is the extended version of a paper accepted to EMNLP 2026 (System Demonstrations)

Regulatory compliance checking - deciding whether a target document satisfies the obligations of a regulation - requires interpreting dense legal text, identifying which provisions apply, and grounding each decision in explicit evidence. We present ARCCS, an end-to-end, automated, agentic, and regulation-agnostic Legal NLP system for compliance checking. ARCCS decomposes raw regulatory text into atomic, traceable requirements and evaluates a target document against them using retrieved evidence, confidence scores, and human-interpretable justifications. This design decouples compliance assessment from any fixed regulatory template or predefined rule set, enabling the pipeline to operate over regulations of varying size and structure. We evaluate ARCCS in two complementary settings. First, in a GDPR policy-document evaluation, LLM-based judges find its decisions and justifications legally and evidentially consistent in up to 96.67% of the assessed cases. Second, on an EU public-procurement benchmark comprising more than 1,200 individual rule checks, the system attains 98.8% accuracy in violation detection. ARCCS is, to our knowledge, the first fully open-source system for end-to-end regulatory compliance checking and auditable report generation.

0.45

AppSec Core: An Ontology for Security-by-Design Software Engineering

cs.SE ·2026-11-12 ·Pedro Farinha
benchmarkcode/data similarity 0.60
Abstract · 24 pages, 4 figures, 6 tables. Supersedes OSF preprint doi:10.17605/OSF.IO/WG8PV

When engineers or coding agents change an application, a dependency, a configuration or a build pipeline, they need to know three things: which security rules apply, how to meet them, and what evidence to leave. Today the answers are spread over many standards, frameworks, laws and lists of weaknesses, and each one uses its own words and level of detail. This paper presents AppSec Core, a model of the security work that engineers do. It says what must be true, which practices get there, which mechanisms do the job and which artefacts show that it was done. It has 259 elements in ten domains. We built it from a security-by-design manual written from practice, and then checked it against 31 outside sources, ten of them about AI systems and agents. Along the way we made three changes, which added five objectives and made one domain wider. We did not need a new domain. Most objectives are backed by several sources: the typical objective is used by six of the twenty technical sources, and 194 of the 202 objectives, practices and mechanisms are linked to items in the sources. For the domain of a change, the model gives back a closed set of typed elements, and for a source, it gives back the objectives that the source points to. We keep the model apart from the manual, the selection rules and the server that delivers the result, and we follow one change, adding a third-party package, through all four. When a code generator received the requirements, the share of security tests passed on a public benchmark went from 77.4% to 93.0%. On a second benchmark, the result was close to that of the benchmark's own "oracle" prompt, which is written with the tests in mind.

0.51

TopoIntent: Compiling Security Intent into Executable, Compliance-Checked Network Topologies

cs.AI cs.CRcs.NI ·2026-08-13 ·Xiaokang Qu, Jianliang Ma, Zao Fan, Tianshu Chu, Tianlong Fan +1 more
benchmark similarity 0.60
Abstract

Enterprise security topology design requires translating business intent, regulatory requirements, and risk assumptions into zones, boundary devices, inter-zone paths, and access-control policies. Existing NetOps automation tools mainly operate after this design is fixed, providing limited support for generating structured security topologies from underspecified natural-language requirements. We present TopoIntent, a system that compiles security intent into executable, compliance-checked network topologies. It uses a schema contract to constrain generation, retrieves reference architectures from a curated template library via dense-vector search, and applies staged fusion for intent-template alignment and security completion. The generated topology is checked against CIS Controls v8.1.2 safeguards visible at the topology layer, while unresolved cases are marked for manual review. Structural gaps are repaired through additive schema-preserving edits. The final topology is exported to Mininet scripts with kernel-level iptables ACLs, enabling executable reachability and allow/deny tests. Because no public benchmark exists for this requirement-to-topology task, we construct an evaluation set from reference security architecture diagrams. The retrieval set contains 22 templates and 44 synthetic intents across five scenarios, while the held-out set contains 7 templates and 14 intents from finance and government scenarios excluded from retrieval. On the held-out set, additive repair improves topology-visible CIS satisfaction from 0.78 to 1.00 in fewer than 1.5 rounds on average, and one feedback round raises the post-ACL policy pass rate from 0.78 to 0.88.

0.04

Towards Tackling Application Logic Flaws through Autonomous Formal-Logic Modeling and Automated Reasoning

cs.CR ·2026-09-09 ·Yiwei Fang, Yichen Liu, Luyi Xing
similarity 0.59
Abstract · 13 pages, 3 figures, 3 tables

Logic flaws pose significant challenges in the design and implementation of modern, semantically rich systems and applications, impacting security, privacy, and trust. These flaws are inherently tied to business-specific semantics and threat models, making their discovery and reasoning difficult and hard to scale. Real-world systems often exhibit diverse application features, complex protocol logic, and domain-specific threat models, necessitating substantial human effort and domain expertise for effective security analysis. In this paper, we introduce LL-Verifier, a novel, automated framework for identifying logic vulnerabilities built on (1) large language models for autonomous modeling, and (2) logic model checkers for rigorous reasoning. LL-Verifier processes natural language inputs, in particular protocol descriptions and security goals, to automatically generate formal logic models and properties expressed in a new logic language built on a generic logic language Maude, optimized for modeling arbitrary application-level semantics. These formal models are then converted into logical state machines, enabling exhaustive, rigorous verification through logic level model checking. This approach streamlines the analysis of diverse, application-level protocols deployed in real-world scenarios, offering automated, exhaustive, and precise reasoning within their logical constraints. We evaluated the high effectiveness, efficiency, and practicality of LL-Verifier by applying it to 27 access control protocols of widely used IoT devices, which come with vendor-specific logic flows and semantics. While LL-verifier tackles a hard problem in application security, i.e., automatic logic flaws discovery, our analysis uncovers a range of sophisticated logic vulnerabilities in IoT protocols and devices with serious security and privacy implications.

0.23