CS Paper Radar

Building Sound Evaluation Practices for Rust-Based Security Frameworks

David Paaßen, Lucas Davi

cs.CR ·submitted 2026-10-14 15:02 UTC · v1

Abstract

Background: Rust is a modern programming language known for its focus on usability, concurrency, and security. However, empirical studies reveal that Rust is still affected by software vulnerabilities, with hundreds of memory safety issues found in Rust programs, including Rust's standard library. As a result, additional tool-assisted security testing is crucial to enhance the security of Rust code. Evaluating such tools, however, is highly complex. While there is substantial research on evaluating C/C++ security frameworks, including test sets, serious issues in published evaluations have been identified, undermining their scientific soundness and reproducibility. In practice, selecting an inadequate implementation (e.g., a fuzzer) can lead to missed vulnerabilities ultimately compromising a program's security. Aims: Our methodology aims to help researchers to conduct scientifically sound evaluations in order to prevent similar issues with the evaluation of security tools and frameworks in the Rust ecosystem. Method: We systematically analyze existing evaluation literature to develop the first sound methodology for Rust based evaluations. Our guidelines provide enough flexibility to support a variety of use cases. To quantify the performance of a security tool, we introduce an automated approach to inject unsafe code coverage oracles, which require no forward porting, avoid overfitting, and allow updating test sets in the future. We also present methods to quantify the diversity of a Rust test set, ensuring that the results are applicable to a large set of existing Rust programs. We practically apply our methodology in our evaluation of the most popular Rust fuzzers. Results: We show the practical applicability of our design and conduct an evaluation with a diverse test set of 20 Rust projects. [...]

Closest papers

Nearest neighbours by meaning across the whole index.

Evaluating and Improving Static Application Security Testing Tools for Rust

cs.CR cs.SE ·2026-10-15 ·Huaien Zhang, Zhengjie Zhao, Yu Pei, Tian Dong, Minxue Pan +2 more
similarity 0.85
Abstract

Rust has become a popular systems programming language in recent years because of its strong guarantees on memory and concurrency safety. Many types of system software have been re-implemented in Rust. However, Rust programs can still suffer from common memory and concurrency vulnerabilities due to unsafe code and misunderstandings of its complex mechanisms. To mitigate such vulnerabilities, developers often rely on static application security testing (SAST) tools. However, the effectiveness of existing Rust SAST tools on real-world vulnerabilities remains underexplored. In this paper, we first conduct an empirical evaluation of mainstream Rust SAST tools using security vulnerabilities identified in real-world Rust applications. We systematically assess their effectiveness, uncover common weaknesses, and analyze the root causes behind them. Motivated by the study, we propose TRUST, a large language model (LLM)-assisted automated testing framework. TRUST automatically analyzes the official Rust reference to derive and implement semantics-preserving transformations. It leverages these transformations to generate mutant programs for evaluating Rust SAST tools and uncovering potential faults within them. Overall, TRUST has revealed 46 faults across six popular SAST tools, 17 of which have been confirmed. In addition, our study yields ten empirical findings and four practical lessons that provide actionable guidance for developing more effective, robust, and maintainable Rust SAST tools.

0.15

C-to-Rust Fallacy: Automatic Refactoring != Memory Security

cs.CR ·2026-09-22 ·Hung-Mao Chen, Xu He, Bo Lu, Xiaokuan Zhang, Kun Sun
similarity 0.80
Abstract

Rust has emerged as the leading system programming language, offering strong memory and type safety guarantees without compromising performance. This positions it as a compelling alternative to traditional languages like C and C++, which are susceptible to memory security bugs. However, manually transforming C to Rust requires in-depth domain knowledge of the Rust language features, which requires significant effort for developers. To address this, tools for automatic C-to-Rust refactoring aim to generate safe Rust code leveraging static analysis and Large Language Models (LLMs). While these tools claim to achieve safety by reducing the unsafe Rust, the correlation with improving security is not clear. In this paper, we conduct a comprehensive empirical study on the reliability, safety, and correctness of various C-to-Rust refactoring methods. Specifically, we evaluate C2Rust-analyze, CROWN, C2SaferRust, and FLOURINE using a dataset of 116 C programs with memory security bugs from the NIST Juliet Test Suite. Based on 464 Rust programs generated by these tools, our evaluation focuses on three key aspects: the compilation correctness of the refactored programs, the effectiveness in mitigating original C bugs, and the tendency to introduce additional Rust bugs. The results indicate that 342 Rust programs fail to compile, 177 Rust programs inherit memory security bugs from the original C programs, and 77 new Rust bugs are introduced. We examine the rationale behind tool design and analyze the root cause of errors across various refactoring methods. Our findings indicate that current automated refactoring tools deliver memory safety as they define it, but not the broader memory security when adopting them.

0.06

Rust for Secure Backend Development: A Critical Review and Extended Vulnerability Comparison with Node.js and Django

cs.PL cs.CR ·2026-08-23 ·Md Zarzees Uddin Shah Chowdhury, Rabib Jahin Ibn Momin, Rifat Shahriyar
similarity 0.78
Abstract · 10 pages, 5 tables. Critically reviews and extends Gasiba and Amburi, "I Think This is the Beginning of a Beautiful Friendship: On the Rust …

The Rust programming language is widely credited with eliminating entire classes of memory-safety and concurrency vulnerabilities, but the security implications of adopting it in practice extend well beyond memory safety. This paper presents a critical review of prior work on Rust's security posture in industrial settings, and extends that analysis in a direction the original study did not cover: backend web development. We first assess the strengths and limitations of the existing vulnerability classification of Rust against C, C++, and Java under the SANS Top 25, OWASP Top 10, and the 19 Deadly Sins of Software Security frameworks, identifying gaps including limited empirical validation, a small interview sample, and the absence of a secure development lifecycle discussion. We then contribute an original comparison of Rust against Node.js and Django using the same three-level classification (Rare and Difficult, Safeguarded, Unprotected), supported by side-by-side code experiments for out-of-bounds writes (CWE-787), use-after-free (CWE-416), and race conditions (CWE-362). Our results indicate that Rust's compile-time guarantees dominate at the systems layer, while managed backend frameworks offer stronger built-in defenses at the application layer, suggesting that Rust adoption in web contexts requires complementary safeguards rather than reliance on language-level safety alone.

0.10

TRACTOR Benchmark for Evaluating C to Rust Translators

cs.SE cs.CRcs.PL ·2026-09-20 ·Hamed Okhravi, Brandt Ogden, Noah Luther, Ian McQuoid, Howard Mak +1 more
benchmarkcode/data similarity 0.77
Abstract

Memory-safety vulnerabilities remain a persistent source of security risk in critical software, much of which is implemented in memory-unsafe languages such as C and C++. Recent advances in programming languages, program analysis, and artificial intelligence have created new opportunities to modernize these legacy systems through automated translation to memory-safe languages such as Rust. The DARPA Translating All C to Rust (TRACTOR) program seeks to develop scalable techniques for translating large C codebases into safe, performant, and maintainable Rust. MIT Lincoln Laboratory serves as the program's independent test and evaluation organization and has developed a standardized benchmark for systematically assessing C-to-Rust translation tools. This report describes the TRACTOR benchmark, including progressively challenging test batteries and larger milestone projects, as well as the supporting evaluation infrastructure and metrics for assessing correctness, safety, idiomaticity, and performance. The benchmark and associated evaluation infrastructure are publicly available to support the broader development and evaluation of C-to-Rust translation technologies.

0.25

Rust's Type Checker Implementation Is Unsound: An Empirical Study on Soundness Bugs in rustc

cs.SE cs.PL ·2026-08-28 ·Yusung Sim, Sukyoung Ryu, Jaemin Hong
top venue similarity 0.76
Abstract · 25 pages, 1 figure. To appear in Proceedings of the ACM on Software Engineering (PACMSE), ISSTA 2026

Rust is claimed to be a type-sound language capable of preventing various undesirable behaviors, including memory bugs. However, rustc, the official Rust compiler, is not immune to defects; it contains soundness bugs, where the compiler accepts programs that should be rejected during type checking. In this work, we present an empirical study of 30 issues that report potential soundness bugs in rustc, collected from the GitHub issue tracker between January 1, 2022 and September 1, 2025. We analyze each issue in depth, focusing on its affected feature, symptom (how the feature is mishandled), consequence (the resulting undesirable behavior), triggering features, community consensus regarding whether it is a bug, and lifecycle, including introduction, discovery, and fix. Furthermore, we investigate existing artifacts, including implementations such as AddressSanitizer, Miri, Chalk, and a-mir-formality, alongside documentation such as the Rust Reference, the FLS, and Rust RFCs to assess their potential as oracles for testing the type soundness of rustc. Our key findings indicate that: (1) Certain soundness bugs, typically triggered by implied bounds or trait objects, compromise memory safety. (2) Sound type checking is challenged by edge cases involving associated types and the interaction between lifetimes and traits. (3) Most bugs persist from the initial introduction of the relevant features and require significant time to be discovered. (4) While AddressSanitizer and Miri can detect soundness bugs that lead to memory bugs, a-mir-formality and Chalk are currently immature despite their potential to identify other bug categories. (5) Existing documentation frequently fails to provide precise explanations of the language semantics.

0.29

UnsafeChecker: Finding Soundness Bugs in Rust Safe Abstractions

cs.PL ·2026-09-09 ·Xizhe Yin, Yaokun Zhang, Yang Feng, Baowen Xu
benchmark similarity 0.75
Abstract

Rust guarantees memory safety without garbage collection through a strict ownership and borrowing system. However, for low-level systems programming, many widely used libraries rely on the unsafe keyword. These libraries encapsulate raw-pointer operations behind safe APIs to form safe abstractions. A single mistake in this internal unsafe code can break its safety contract, rendering the abstraction unsound and allowing safe clients to trigger undefined behavior. Detecting these potential soundness violations is challenging. Existing static analysis tools for C/C++ ignore Rust-specific safety contracts, while current Rust tools lack the deep semantic modeling required to track the contexts that raw pointers erase. To address this gap, we present UnsafeChecker, a compiler-integrated static analysis framework for detecting potential soundness violations in Rust safe abstractions. UnsafeChecker analyzes Rust MIR using a flow-sensitive abstract interpretation that maintains a shared state with three components: ownership, object validity, and layout. Each warning rule consumes the subset of facts needed for the corresponding Rust safety obligation. UnsafeChecker reports both instruction-level undefined behavior and boundary-level contract violations that may escape through safe APIs. We evaluate UnsafeChecker on a benchmark of 46 RustSec vulnerabilities, which contain 53 ground-truth bugs. UnsafeChecker outperforms several state-of-the-art tools, detecting 32 CVEs and covering 36 bugs (67.9% recall) with 51.6% alert-level precision. Furthermore, in a large-scale scan of real-world crates on crates.io, UnsafeChecker uncovered 114 previously unknown bugs across 83 crates, with 45 confirmed and 27 already fixed by maintainers.

0.09

Modular Responsiveness Verification of Rust Async Runtimes

cs.PL ·2026-10-06 ·Yanze Li, Ivan Beschastnikh, Alexander J. Summers
published similarity 0.71
Abstract · To appear at ESOP 2027, preprint with appendix

Asynchronous (async) programming is a popular paradigm for managing concurrency. Languages that provide async support typically have a runtime to manage asynchronous executions. These runtimes are critical infrastructure, yet verifying them has received little attention. One reason is that a property that users care most about from an async runtime is a liveness property: tasks submitted to the runtime eventually make progress. Verifying liveness is challenging for libraries that are both concurrent and highly optimized. We present a lightweight and modular proof technique for verifying eventual progression guarantees for Rust async runtimes. We describe this technique in the context of a simple language based on Rust and Rust's async model. We then realize this proof technique as a set of static analyses for Rust and use these to verify eventual progression of several key components of multiple Rust async runtime implementations.

0.19

Crustify: Repository-Level Generation of Safe Rust Bindings with LLM Agents and Coverage-Guided Testing

cs.PL cs.AIcs.CRcs.SE ·2026-11-07 ·Marius Momeu, Marius Arvinte, Cory Cornelius, Koushik Sen
similarity 0.71
Abstract

Wrapping C code in a safe Rust interface lets Rust applications use it without writing unsafe code, until the code itself is migrated to Rust. Such bindings are tedious and error-prone to write, however: their internals use unsafe code that the Rust compiler does not check, so a single mistake can introduce undefined behavior (UB), and they have so far been written mostly by hand. We present Crustify, an agentic co-translator that generates safe Rust bindings for repository-level C libraries. Crustify schedules a target's types and symbols into dependency-ordered waves of independent batches that agents bind concurrently. Its agents derive the ownership and lifecycle contract of every pointer and type they wrap, check each binding with coverage-guided equivalence and UB tests under sanitizers and Miri, and review every wave. They build on three tools that we contribute: (a) ffibox, Rust primitives for C/Rust interop that mirror Box, Arc, and Vec for C objects without ever forming a reference to memory that C may alias; (b) Wavefront, a neuro-symbolic static analyzer that orders a target's items by their dependencies, including those that casts and type-erased pointers hide; and (c) ffimap, a rustc-based analysis that maps the unsafe surface of bindings. On libgit2 and ffmpeg, with one developer in the loop, Crustify generated safe bindings for 41.9% and 42.8% of their public APIs in about 51.9 and 58.9 hours. On the interface consumers are expected to use, only 1.6% and 0.4% of the bindings' public functions are unsafe, and only 0.1% and 0.6% have a raw pointer in their signature. Their tests cover 98.92% and 99.07% of the Rust code and 45.95% and 67.20% of libgit2 and of ffmpeg's libavutil, and the reviewers fixed 8 and 13 defects that caused UB reachable from Rust. To our knowledge, this is the first work to automate the generation of safe Rust bindings for C libraries.

0.09