CS Paper Radar

Barriers to Using Static Application Security Testing (SAST) Tools: A Literature Review

Zachary Wadhams, Clemente Izurieta, Ann Marie Reinhold

cs.SE cs.CR ·submitted 2026-09-01 04:04 UTC · v1
top venuesurveypositionhighly cited

Abstract

Developers face a challenging problem with no clear solution. Modern software breaches can wreak havoc on businesses and individuals alike. With code vulnerabilities being a leading cause, securing applications must be a priority for developers. Static Application Security Testing (SAST) has the potential to harden applications by assisting in the identification and resolution of security vulnerabilities. Despite this, many development teams have not adopted SAST tools into their environment. In this paper, we survey the recent literature to uncover why some developers are apprehensive towards SAST and identify what specific problems they encounter when using it. We found a variety of usability problems developers face when using SAST. Some are inherent of the tool and ultimately require some level of developer investment while others are tool shortcomings that SAST tool creators must address. Ultimately, we argue that in order to drive widespread adoption and consistent SAST usage, developers will need to embrace that some investment is required. Simultaneously, developers will be more likely to integrate SAST tools into their workflows if the creators of SAST tools simplify many aspects related to tool usage. Surmounting the primary obstacles preventing the adoption of SAST requires full consideration of both the technical and human factors.

Comment
6 pages, 4 figures, published at 2024 39th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW)
Journal ref
39th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW '24), Sacramento, CA, USA, 2024, pp. 161-166
DOI
10.1145/3691621.3694947

Closest papers

Nearest neighbours by meaning across the whole index.

Automating Static Code Analysis Through CI/CD Pipeline Integration

cs.CR cs.SE ·2026-09-01 ·Zachary Wadhams, Ann Marie Reinhold, Clemente Izurieta
top venuehighly cited similarity 0.81
Abstract · 7 pages, 3 figures, published in 2024 IEEE International Conference on Software Analysis, Evolution and Reengineering - Companion (SANER-C),…

In the contemporary landscape of software devel-opment, securing sensitive data is paramount to safeguarding organizational reputation, preventing financial losses, and pro-tecting individuals from identity theft. This paper addresses the pervasive challenge of identifying and rectifying security vulnerabilities early in the development process, emphasizing the role of Static Application Security Testing (SAST) tools. While SAST tools play a crucial role in detecting vulnerabilities, widespread adoption has been hindered by usability issues, including high false positive rates and a lack of native pipeline support. This paper proposes a novel, generalized, and automated process for aggregating SAST tool outputs and integrating them into developers' familiar issue-tracking software. The process streamlines the identification and communication of security vulnerabilities during the development lifecycle, facilitating more efficient remediation efforts. We demonstrate the successful implementation of the proposed process with the SonarQube SAST tool in a GitLab-based development environment. Developers were positive about the structured implementation, real-time feedback, and proactive vulnerability management. However, despite some challenges such as a potential learning curve and tradeoffs between secure coding and workflow disruption, the overall positive impact on security awareness and responsiveness suggests that the proposed process holds promise in enhancing the security posture of software development practices

0.74

Investigating Developer-Reported Software Security Testing Challenges

cs.SE cs.CR ·2026-09-10 ·Md Erfan, Ahmed Ryan, Md Rayhanur Rahman
similarity 0.71
Abstract · 34 pages

Software security testing (SST) is essential for identifying vulnerabilities and improving software security, but developers often face practical challenges when selecting tools, configuring test environments, interpreting scanner outputs, testing authentication workflows, and acting on reported vulnerabilities. This study empirically characterizes developer-reported SST challenges in Stack Overflow (SO) discussions and examines their prevalence, difficulty, temporal evolution, co-occurrence, and taxonomy stability. We analyze 17,743 SO questions collected using SST-related keywords. Through manual labeling, we identify 582 SST-related questions and construct a taxonomy of 8 categories and 31 subcategories. We then analyze these challenges using prevalence, difficulty, correlation, temporal trend, co-occurrence, and held-out stability analyses. The results show that developers frequently discuss security finding interpretation and reliability, security testing guidance and tool selection, authentication and authorization testing, and security tool integration and automation. Validation-related challenges often require more technical context and longer resolution time. Security finding interpretation and remediation-related actionability show increasing trends over time. Co-occurrence analysis shows that false positives frequently appear with explainability, while integration challenges often appear with tool suggestions and documentation/resources. Overall, developer-reported SST challenges extend beyond vulnerability detection and include workflow, configuration, interpretation, and remediation concerns. These findings can help researchers, practitioners, educators, and tool providers improve SST usability, documentation, result interpretation, and remediation support.

0.25

Evaluating and Improving Static Application Security Testing Tools for Rust

cs.CR cs.SE ·2026-10-15 ·Huaien Zhang, Zhengjie Zhao, Yu Pei, Tian Dong, Minxue Pan +2 more
similarity 0.70
Abstract

Rust has become a popular systems programming language in recent years because of its strong guarantees on memory and concurrency safety. Many types of system software have been re-implemented in Rust. However, Rust programs can still suffer from common memory and concurrency vulnerabilities due to unsafe code and misunderstandings of its complex mechanisms. To mitigate such vulnerabilities, developers often rely on static application security testing (SAST) tools. However, the effectiveness of existing Rust SAST tools on real-world vulnerabilities remains underexplored. In this paper, we first conduct an empirical evaluation of mainstream Rust SAST tools using security vulnerabilities identified in real-world Rust applications. We systematically assess their effectiveness, uncover common weaknesses, and analyze the root causes behind them. Motivated by the study, we propose TRUST, a large language model (LLM)-assisted automated testing framework. TRUST automatically analyzes the official Rust reference to derive and implement semantics-preserving transformations. It leverages these transformations to generate mutant programs for evaluating Rust SAST tools and uncovering potential faults within them. Overall, TRUST has revealed 46 faults across six popular SAST tools, 17 of which have been confirmed. In addition, our study yields ten empirical findings and four practical lessons that provide actionable guidance for developing more effective, robust, and maintainable Rust SAST tools.

0.15

Characterizing GitHub Security Actions Alerts For Web Application Tutorials

cs.SE ·2026-11-18 ·Bhagya Chembakottu, Martin P. Robillard
similarity 0.68
Abstract

The high frequency of cybersecurity incidents involving web applications highlights the importance of secure software development practices. Static Application Security Testing (SAST) tools help identify potential vulnerabilities in source code without executing the code. Integrating SAST tools into GitHub Security Actions enables security alerts to be reported during development. However, the characteristics of these alerts can make them difficult for developers with limited security expertise to understand, prioritize, and address. Many tutorials for learning web application development also involve source code with security implications. We investigated the properties of SAST alerts relevant to this learning context. We analyzed 6,609 alerts from four default GitHub Security Actions configurations across 60 Django and Spring tutorial repositories. We evaluated the alerts along four dimensions: alert volume, alert relevance, conceptual breadth, and remediation usability. Of the 6,609 alerts, 3,750 (56.7\%) targeted tutorial-related files, while 2,859 (43.3\%) targeted other repository files. The configurations triggered between 34 and 646 distinct rules, and the five most frequently triggered rules accounted for 25.2\%--68.2\% of alerts, depending on the configuration. Across configurations, 0.4\%--32.1\% of alert messages named concrete framework or code constructs, and 0\%--8.1\% met our operational criteria for remediation specificity. Overall, default GitHub Security Actions report a diverse set of security alerts, but these alerts do not consistently focus on tutorial code or provide concrete guidance for addressing the reported issues.

0.16

Code-Grounded Agentic Explanations for Static Analysis Vulnerability Findings

cs.SE cs.AIcs.HC ·2026-11-09 ·Oshando Johnson, Ranjith Krishnamurthy, Rohith Kumar Shanmuganathan, Goran Piskachev, Eric Bodden
code/data similarity 0.67
Abstract

Static application security testing (SAST) tools surface vulnerabilities early and their verdicts are rule-based and auditable. Yet developers routinely dismiss findings they cannot interpret. Large language models can supply the missing explanation, but designs that also hand the model the detection give up the auditability that makes static analysis defensible. We keep detection with the analyser and give the model only the explaining. A preliminary user study with 20 participants and a workshop with 19 practitioners yielded five requirements. Guided by these we redesigned SAFE, an open-source IntelliJ IDEA plugin that explains findings from any analyser reporting in the Static Analysis Results Interchange Format. It tiers explanations by demand, ties them to the reported data flow, and replaces the single-prompt backend with an agent that must consult the repository and the rule that fired before answering. A within-subjects crossover experiment with 24 participants compared the redesign against its predecessor. Perceived usability rose from 77.3 to 85.0 on the System Usability Scale. Actionability gained most among the explanation-quality items, the aspect both earlier studies flagged as weakest, and the odds of a correct remediation were 2.6 times higher. Triage accuracy and time per finding were indistinguishable, which we read as uninformative rather than as equivalence. Trust was the one measure that did not move, and the closing interviews indicate that grounding made findings easier to understand rather than easier to believe. Participants also drew a boundary no requirement anticipated, wanting the tool to find and explain while they judge and apply.

0.48

How Well Do SAST Tools Work Across Languages? An Empirical Study Using OCPP Implementations

cs.CR cs.SE ·2026-11-10 ·Zachary Wadhams, Emma Sheppard, Dalton Arford, Clemente Izurieta, Ann Marie Reinhold
published similarity 0.66
Abstract · 6 pages, 2 tables, Author-accepted manuscript. Published at IEEE CSR 2026

As electric vehicle (EV) adoption accelerates globally, securing electric vehicle charging station (EVCS) software becomes increasingly critical. The Open Charge Point Protocol (OCPP), widely adopted for EVCS management, places significant responsibility on developers to securely implement specifications that are subject to multiple interpretations. The variety of OCPP implementations across different programming languages provides a unique opportunity to evaluate security tooling in a real-world, multi-language context where implementation variability may introduce vulnerabilities. This study provides the first empirical evaluation of Static Application Security Testing (SAST) tool performance across multiple programming languages using OCPP implementations as a real-world test corpus. We applied both comprehensive multi-language tools (SonarQube and Semgrep) and language-specific analyzers to twelve OCPP 1.6 implementations--each written in a distinct programming language--to examine how detection patterns vary across language ecosystems. Total issues ranged from just two in Rust to 1,689 in Java. Mature programming languages like Java yielded substantially more findings than newer languages like Rust, likely because established languages have more developed SAST tool ecosystems rather than inherently worse code quality. These results suggest that developers should consider available SAST tooling when selecting implementation languages for security critical systems.

0.36

Combining LLM and Dynamic Analysis for Universal Testing of Static Application Security Testing Tools

cs.CR cs.SE ·2026-10-15 ·Huaien Zhang, Yu Pei, Hao Chen, Shin Hwei Tan
similarity 0.65
Abstract

Static application security testing (SAST) tools help detect potential software vulnerabilities, but they themselves contain faults undermining reliability and usability. Automated testing can address these issues, yet current techniques suffer from limited program transformation diversity and lack mechanisms to verify generated variants, leading to reduced fault detection and high manual effort. We present LLM4SAST, the first universal framework detecting faults in SAST tools using large language models and dynamic feedback. Our key insight is leveraging LLMs to understand and transform code in diverse programming languages. We design novel metamorphic relations with transformations to guide variant generation, using an LLM to produce multi-language variants that satisfy these relations. To reduce erroneous variants and identify unique faults, we propose two heuristics: dynamic-feedback-driven selection and program-differencing-based clustering. For each metamorphic relation, a report-centric test oracle detects variants that fail tests and identifies underlying faults by differentially analyzing SAST reports of the seed program and its variants. LLM4SAST identified 66 unique faults across six SAST tools, with 32 confirmed or fixed after being reported to developers, demonstrating effectiveness. The proposed heuristics eliminate 47.8% of false positives, and LLM4SAST detects 3.4 times more faults while requiring only 39.8% of the variants compared to existing approaches.

0.19

Propagation Model for SSC attacks: Why SBOM (tools) don't tell the whole truth

cs.CR ·2026-09-04 ·Ljubica Grgic, Lazar Maksimovic, Pavel Laskov
publishedpositioncode/data similarity 0.65
Abstract

Ensuring security of software supply chains (SSC) is indispensable in today's world of modern software practices. SBOM (tools) have been introduced as relevant building blocks to ensure the transparency of SSCs. However they have serious limitations in practices as their vulnerability detection and interpretation capacity is not sufficient to explain exploitability effects that can propagte through the whole chain. To address this gap, we propose a propagation-centred approach to SSC security and introduce a four-stage propagation model. We empirically evaluate four open-source SBOM tools against each stage using three projects and Log4j vulnerability as our test case. Our results show that current SBOM tools systematically support only Stage 1 (Structural Exposure) and Stage 2 (Vulnerability Class Presence) while Stage 3 (Code Reachability) and Stage 4 (Taint Path Analysis) require capabilities absent from the SBOM ecosystem. We argue that putting propagation effects at the centre of SSC security research is essential to prevent cyber risk evolving into systemic risks. Our research findings contribute to a future research and design of modern SSC security tools.

0.67