CS Paper Radar

A Survey of Security Research for Operating Systems

Masaki Hashimoto, Ruo Ando, Toshiyuki Maeda, Hidehiko Tanaka

cs.CR ·submitted 2026-10-08 08:20 UTC · v1
top venuesurveyunusualhighly cited

Abstract

In recent years, information systems have become the social infrastructure, so that their security must be improved urgently. In this paper, we introduce the results of the survey of virtualization, operating system verification and access control technologies in association with the design requirements of the reference monitor. Additionally, we show the prospects and challenges for each technology.

Journal ref
IPSJ Transactions on Advanced Computing Systems (ACS), Vol. 5, No. 2, pp. 51-62, March 2012

Closest papers

Nearest neighbours by meaning across the whole index.

safeL4: Rethinking Safe-Language-Based Single Address Space Microkernels

cs.OS cs.CRcs.PL ·2026-10-26 ·Zheyuan Xu, Xiaobei Zhang, Zihan Li, Zekai Nie, Guangtao Zhu +5 more
top venue similarity 0.53
Abstract · ASPLOS'27

Microkernels provide good features such as high security, safety, and flexibility, but their reliance on hardware-based isolation leads to heavy performance overhead, limiting their widespread adoption. Although constructing single-address-space microkernels based on safe language-based isolation is a promising approach to addressing the performance issue, this approach still faces challenges in maintaining the principle of minimality and enforcing capability-based, fine-grained access control over system resources. We propose safeL4, an approach to constructing microkernels based on safe languages. It guarantees minimality through low-level abstractions for unsafe operations, and implements a capability system based on the safe language's object capabilities to achieve fine-grained access control over all operations on system resources. We implement a prototype system in Rust, and our evaluation results show that the primitives provided by safeL4 are, on average, 1.9$\times$ faster than those of seL4.

0.27

Breaking Ambient Trust: In-Network Per-Process Access Control Against Lateral Movement

cs.CR cs.NI ·2026-08-30 ·Osama Bajaber, Bo Ji, Peng Gao
hot direction similarity 0.52
Abstract

Enterprise networks remain vulnerable to Advanced Persistent Threats (APTs), where adversaries gain an initial foothold and move laterally across the network, accumulating access permissions hop by hop to reach critical targets. Existing network defenses cannot track user movement at the process level across the network; instead, they grant ambient trust to all processes within a host. As a result, once a host is compromised, malicious processes inherit the victim's permissions, thereby expanding the attacker's access scope and enabling further lateral movement. To address this gap, we present NetZone, an in-network access control that confines each user process to a fixed access scope that persists as the user moves across the network. NetZone introduces a new abstraction, called AccessScope, which represents a lightweight access capability bound to the user's processes. Each AccessScope encodes the set of hosts a user identity is authorized to access and is embedded in the process's outgoing network traffic for validation before reaching its destination. As users pivot across hosts, AccessScope propagates with their traffic, rebinds to the receiving process, and persists across hosts. This ensures that regardless of network location, the user's processes are consistently governed by their bound AccessScope and their access permissions remain unchanged. To handle the high volume of network traffic generated by processes, we develop a data-plane co-design that integrates programmable switches with eBPF. NetZone employs a set of in-network optimizations and lightweight AccessScope persistence techniques to inspect the embedded AccessScope on the fly, enabling line-rate processing of high traffic volumes with negligible latency overhead. Our extensive evaluations show that NetZone can effectively defend against sophisticated attack scenarios without introducing noticeable overhead.

0.30

Building the Truman Show: A TrustZone-Based Framework for Lightweight Out-of-band Kernel Security Monitoring

cs.CR cs.SE ·2026-08-30 ·Zhenling Duan, Pan Dong, Renshuang Jiang, Xiaoxiang Fang, Bao Li
similarity 0.52
Abstract

The increasing number of vulnerabilities in operating systems, together with sophisticated kernel-level threats (e.g., rootkits), has weakened the effectiveness of traditional in-kernel protection mechanisms. Since these defenses operate at the same privilege level as the kernel, they share the same attack surface and can be bypassed once the kernel is compromised. Isolation-based security approaches provide stronger protection by separating security logic from the kernel, but strict isolation often introduces semantic gaps that limit system visibility and hinder timely threat detection. In this paper, we present LOOM, a lightweight out-of-band operating system monitoring architecture built on ARM TrustZone. By leveraging TrustZone's hardware-enforced isolation, LOOM establishes a tamper-resistant monitoring environment independent of the kernel. To bridge the semantic gap, we design a lightweight semantic reconstruction mechanism in the Secure World. It selectively captures the states and behavioral patterns of critical kernel objects, such as process control blocks and kernel modules. Additionally, LOOM introduces a dual-stage hazard prevention mechanism that combines atomic memory protection with an interrupt-driven adaptive agent to detect and mitigate kernel rootkit activities. An address translation cache is further incorporated to optimize repeated address access and reduce monitoring overhead. Overall, we develop a multi-layered collaborative architecture with platform, functional, and auxiliary layers for secure and efficient kernel monitoring. A prototype of LOOM has been implemented on the Phytium D2000 platform. Experimental results indicate that LOOM incurs negligible overhead while maintaining a strong monitoring capability. Furthermore, a security capability analysis based on CVE cases demonstrates that LOOM can detect and mitigate various kernel attacks.

0.18

TPMSpy: Validation of Measured Boot Systems by Low-Level Tracing of TPM Usage

cs.CR ·2026-09-04 ·Roman Lacko, Petr Svenda
top venue similarity 0.52
Abstract · 20 pages, 4 figures

Measured Boot extends trust in a booted system by recording cryptographic measurements of executed software and system state into a Trusted Platform Module (TPM), enabling subsequent verification through remote attestation. Although this mechanism is increasingly deployed in contemporary operating systems, its practical security depends on whether implementations measure the expected components under the expected conditions, yet this is not checked systematically. We propose a platform-agnostic method for analysing low-level TPM usage at the level of virtualized system--TPM interactions. It enables independent reconstruction and validation of the TPM Event Log without relying on the quoting mechanism itself. Because it does not depend on implementation details, it is applicable to both open and closed systems. We demonstrate the method on both Linux and Windows and conduct a systematic longitudinal analysis of Linux systems with systemd versions 245--258 (2020--2025), examining how Measured Boot usage evolved and observing wide divergence. No single usage pattern emerged amongst systems, warranting customized analysis. The analysis identifies undocumented behavioural changes, reveals inconsistent measurements of user-space systemd services, which prevent reliable remote attestation and LUKS disk decryption on such systems.

0.37

SysComb: Fine-Grained Transparent System Call Filtering for Attack Surface Reduction

cs.CR ·2026-08-27 ·Matthew Rossi, Marco Abbadini, Michele Beretta, Dario Facchinetti, Stefano Paraboschi
similarity 0.51
Abstract

Restricting the system calls available to applications shrinks the kernel's attack surface and greatly mitigates the impact of compromised programs. Recent approaches showcase techniques to generate system call filters, however, all existing solutions require either kernel or application modifications to activate them at runtime. This is intrusive, error-prone, and often impractical, especially when the code is maintained by external parties. This paper presents SysComb, a novel eBPF-based solution to enforce temporally-specialized system call filters based on the application state, without requiring any modification to the application or the kernel code, and thus addressing the above limitations. Moreover, SysComb lets the developer choose between two distinct enforcement strategies: seccomp-like, ensuring no new privileges are gained after a state transition is performed, and least-privilege, which applies to each state the most restrictive filter. We evaluated SysComb using widely used software, showcasing accurate state-aware system call filtering and an overhead comparable to built-in kernel solutions, demonstrating the practicality of our approach.

0.31

Ring-0 by Default: Agents as Operating Systems (AaOS)

cs.CR cs.OS ·2026-10-24 ·Xiang Li, Aichun Huang, Zuyao Xu, Fasheng Miao, Zheli Liu
position similarity 0.51
Abstract · 30 pages, 5 figures

LLM agents now read and write files, start processes and subagents, fetch network content, load third-party extensions, and save and resume state, which are the responsibilities of an operating system. They gained these functions within about three years, but not the protections that operating systems built over five decades: whatever the model outputs, whether it follows the user, a tool result, or a downloaded skill, runs with the full authority of the host user, in effect at ring 0. Prior work builds an operating system for agents (OS4A); we argue that a deployed agent already is like an operating system (AaOS) and analyze it as one. Taking the mechanisms of standard OS textbooks as a checklist, we map 116 mechanisms in 19 subsystems to their agent counterparts, from product documentation, source code, and the literature, and label each as isomorphic, degraded (function present, protection dependent on the model), missing, mutated, or novel. Only 11 are isomorphic and 65 are missing, most of them protections: privilege separation, resource quotas, transactions, and tamper-proof audit. Existing defenses confine the whole agent or review its extensions and leave this interior unprotected, and by Anderson's reference-monitor criteria (non-bypassable, tamper-proof, verifiable), rules written in the prompt satisfy none. We test the map with thirteen experiments on Claude Code, Codex CLI, and Gemini CLI, covering 25 mechanisms whose enforcement documentation cannot settle: 16 match the analysis, 7 gain a more precise description, and 2 change verdict. Checks sit in single tools, so the shell bypasses them; all three CLIs load forged transcript records on resume; and task control, sub-agent permissions, and model-based monitors are weak. We identify five problems for which no OS defense transfers and derive a prioritized list of open problems from the gaps of the map.

0.28

Reconfiguring System-Level Protection Through Bounded Delegation in RISC-V MCUs

cs.CR cs.AR ·2026-11-09 ·Luís Cunha, José Martins, Manuel Rodríguez, Tiago Gomes, Uwe Moslehner +5 more
similarity 0.51
Abstract

MCU-based embedded and cyber-physical systems increasingly require protection policies to change at runtime as resources, software, and system composition evolve. Yet widely used COTS MCU protection mechanisms are still largely designed around statically established policies. While they can restrict which domains modify protection primitives, they generally do not constrain what policy an authorized domain may create. As a result, domains may expand protection rules beyond their intended scope or interfere with boundaries established by other domains. We identify bounded configuration authority as a missing primitive for secure dynamic policy evolution and introduce an ownership-aware model in which explicit ownership and rule containment bound both configuration authority and recursive delegation. Changes to active protection-rule ranges are validated before commit, preserving protection boundaries without extending the normal access-checking datapath. We implement the model in an open-source RISC-V World Checker integrated into a CVA6-based SoC. Our FPGA evaluation shows no degradation in access latency or operating frequency, with 7.1% LUT and 1.2% FF overhead for eight rules. In an OTA update case study, direct reconfiguration under delegated authority reduces policy-transition latency by 89.5% over hypervisor mediation and 94.6% over trusted firmware mediation, while preserving established authority boundaries. All artifacts will be open-sourced to support the evolution of RISC-V specifications and system-level protection designs.

0.31

More Granular, Less Trust: Enforcing Intra-Process Isolation with Arm CCA in an Untrusted Management Environment

cs.CR ·2026-08-20 ·Shiqi Liu, Zhouqi Jiang, Jie Wang, Wei Zhou, Kun Sun +2 more
top venuehighly cited similarity 0.51
Abstract · Published in IEEE Transactions on Information Forensics and Security (TIFS)

With the increasing adoption of confidential computing, security-sensitive applications are often deployed in confidential virtual machines (CVMs), which reduce reliance on third-party cloud providers. However, privilege attacks originating from the OS remain a significant threat in these environments. Existing finer-grained isolation schemes, such as SHELTER, provide process-level protection but are still vulnerable to intraprocess attacks and potential collusion between the OS and intra-process adversaries. Many current intra-process isolation techniques continue to depend on the OS to manage and enforce isolation domains, leading to a large Trusted Computing Base (TCB). This gap highlights the need for more granular, less trust-dependent confidential computing solutions. In this paper, we present CCAegis, a system that extends the Arm Confidential Compute Architecture (CCA) to enforce intra-process isolation of sensitive data and operations, safeguarding them from both intraprocess adversaries and the OS. We employ static analysis to track the flow of sensitive data and identify functions that handle such data. Permission-switching instructions are inserted at the function call and return points, adjusting permissions via the Granule Protection Table (GPT) to ensure that only designated functions can access the isolated data. Notably, CCAegis places trust solely in the Secure Monitor, which configures the GPTs and manages domain switching, thereby minimizing the TCB. We implemented CCAegis on both an official emulator and a real development board to assess its performance. Our experimental results show that CCAegis effectively isolates sensitive data and operations, with performance overheads ranging from 1.01x to 1.43x compared to the original version across real-world cryptographic workloads.

0.57