CS Paper Radar

Prioritizing Microservice Requests via Retries

Ofek Lavi, Jhonatan Tavori, Hanoch Levy, Anat Bremler-Barr

cs.NI cs.CRcs.DC ·submitted 2026-10-14 17:51 UTC · v1

Abstract

Cloud services rapidly shift towards independently-managed microservices architectures. This independence may lead to temporary microservices miscoordination, resulting in overloads and rejections of valuable transactions. This weakness may further be exploited by attackers to launch overload attacks inflicting performance and cost damages. Retry operations are a key machinery aiming at overcoming seldom rejections. However, during overload periods they: a) Fail protecting valuable transactions from rejections, and b) Trigger large-scale performance-degrading retry storms. We propose simple modifications in retry operations, overcoming these deficiencies by transforming retries into a rejection prioritization mechanism, DiffTry (Differential Retries). It is based on a simple, lightweight, distributed and stateless design and requires neither overload sensing nor microservices coordination. We develop an analytical model capturing DiffTry's effectiveness for two services in tandem, then generalize it to networks of services. Our theoretical framework lets practitioners quantify, for a given network, the resulting prioritization and rejection rates, and yields formal guarantees that DiffTry substantially reduces the impact of both incidental and adversarial orchestrated overloads. We validate these predictions in simulation and on a live Kubernetes deployment, where DiffTry is a simple drop-in service-mesh plugin requiring no application changes; the measured rejection closely matches the analysis.

Closest papers

Nearest neighbours by meaning across the whole index.

Retry Amplification in Distributed Systems: A Systematic Analysis of Retry Policies and Their Role in Cascading Failures

cs.SE ·2026-08-26 ·Rishabh Mehan, Jasmit Kaur Saluja
publishedcode/data similarity 0.68
Abstract

Retry mechanisms are a standard component of resilient distributed systems, but their collective behavior, when every tier in a call path retries concurrently, is less well understood than the per-client guidance that produced them. This paper introduces the retry amplification factor (RAF), a metric quantifying the additional request volume that retry policies generate during partial failures. In a study of 200 open-source Python microservice projects, explicit retry logic is detected in 11.5%, and an audit of our own false negatives places true prevalence near 41%. Among the projects detected, 60.9% contain at least one configuration without backoff, and after manual verification exactly one of 113 production configurations randomizes its delay. We then evaluate these policies in simulation (n = 100 trials per strategy). Under correlated failures, a naive standard retry policy reduces the success rate from 55.4% to 41.5% relative to performing no retries at all. We catalog five recurring anti-patterns, propose Adaptive Retry Budgeting (ARB), and show that budget-constrained retries maintain success rates close to the no-retry baseline while still recovering from transient faults. These results indicate that retry behavior should be designed as a system-level property rather than configured locally at each call site.

0.66

SafePar: Monitoring Asynchrony in Microservices

cs.PL cs.FLcs.NI ·2026-09-27 ·Karuna Grewal, P. Brighten Godfrey, Justin Hsu, Umang Mathur
hot direction similarity 0.63
Abstract · 27 pages, 6 figure

Modern cloud applications are built from loosely-coupled microservices that coordinate through well-defined APIs to service user requests. A single API request often triggers multiple downstream API calls, some executed sequentially and others spawned asynchronously in parallel. To certify safe and secure inter-service interactions in such applications, security and compliance teams must enforce policies not only over nested call/return structure, but also over the parallel structure of an execution: which calls may run concurrently, how many parallel branches can be spawned, and what combination of branch outcomes are allowed. However, existing runtime enforcement mechanisms typically model executions as sequential or purely nested traces, and cannot capture the parallel structure introduced by asynchronous API calls. Furthermore, since application implementations may not be accessible to security and compliance teams, the policy enforcement mechanism should be decoupled from the service implementation. We introduce SafePar, a specification and monitoring framework for policies over concurrent microservice executions. A SafePar policy constrains both the order of API calls and their series-parallel structure. To support seamless deployments, each policy is compiled into a series-parallel visibly pushdown automaton, a new model of computation we propose in this work, that drives a distributed runtime monitor implemented on top of the servicemesh layer. Our technique is blackbox and non-invasive: it requires no access or changes to the service implementation. Our experiments show that SafePar enforces rich concurrency-aware policies while incurring only millisecond-scale latency overhead.

0.25

Procrustes-Bering-Sheaft: Automated Model Discovery and Virtual Crash Tests for Microservices

cs.SE cs.DCcs.PF ·2026-10-22 ·Anatoly A. Krasnovsky
similarity 0.61
Abstract · Submitted to the ICSE 2027 Tool Demonstration and Data Showcase track

Procrustes-Bering-Sheaft automates lightweight model discovery and virtual crash tests for microservices. Each test estimates operation availability by Monte Carlo sampling of service and communication failures. Procrustes checks deployment and tracing evidence; Bering discovers dependencies and incorporates replica declarations; Sheaft evaluates declared operation-success conditions. Independent command-line interfaces exchange reusable, versioned artifacts. The toolchain extends an earlier graph-based approach with mandatory-call contracts, failure-tolerance sweeps, baseline comparison, component-importance analysis, and CI/CD gates. We demonstrate the workflow on one HTTP operation. Six controlled execution states exposed two mismatches between reachability and observed success; an explicit required-call predicate matched all six. Two modelled one-replica additions produced different decisions: recommendation retained an availability-target violation, whereas catalog restored the baseline estimate of 0.979900 and grid tolerance of 0.12. Analytical oracles and controlled tests verify the implementation; archived replay checks numerical continuity with two earlier studies.

0.33

RESTberus: Trace-Aware Benchmarking of Automated REST API Testing Tools Across Microservice Dependencies

cs.SE ·2026-11-03 ·John Toniutti, Massimiliano Baldo, Marino Miculan, Vincenzo Riccio
similarity 0.59
Abstract

Existing REST API testing benchmarks focus on isolated services, overlooking failures arising from interactions with downstream dependencies. In microservice systems, response bodies and entry-point operations alone cannot reliably distinguish failures, compromising tool comparisons. We present RESTberus, a Kubernetes-based benchmarking platform for black-box REST API testing tools on deployed microservice systems. By complementing API-level evidence with distributed traces, RESTberus can distinguish failures manifesting in the target service from those manifesting in downstream dependencies. Across six tools on two microservice benchmarks, our trace-aware deduplication strategy achieves 87.1% accuracy, outperforming baselines from prior work by 10-13 percentage points. Our empirical study shows that all tools expose downstream failures, accounting for 11.7% of estimated unique failures on average, which isolated-service benchmarks cannot expose. On single-service benchmarks, RESTberus produces statistically equivalent results to RESTgym, demonstrating that integration-level assessments can preserve compatibility with existing tools and comparability with prior evaluations.

0.27

GateDrain: Availability Attacks and Admission-Side Defense for Confidence-Gated Edge-Cloud Inference

cs.CR ·2026-09-27 ·Zonghua Gu, Julian Singh-Smith, Junlin Liao, Di Liu
similarity 0.59
Abstract

Confidence-gated edge--cloud inference accepts confident local predictions and offloads uncertain inputs to a stronger cloud model. We show that this routing decision creates an availability attack surface. We call this attack \emph{GateDrain}: bounded input perturbations lower calibrated confidence and redirect requests that would otherwise be answered locally into a shared cloud queue, without increasing the application request rate. Because escalated requests share a cloud service, an increase in per-request cloud demand can move a near-capacity deployment across a queueing knee, causing disproportionate tail-latency degradation for benign users. We evaluate white-box, transfer, decision-only, universal, and multi-gate attacks on the public EdgeBoost artifact. A fixed-application-volume comparison isolates the effect of confidence manipulation from added client traffic, while perturbation-budget and arrival-process sweeps show that the queueing transition persists across several workload models but its amplification depends on the operating point. Adaptive attacks also defeat the evaluated training-free preprocessing defenses. To contain the resulting cloud demand, we evaluate Bounded Escalation, which combines per-source admission budgets, protected capacity, and non-preemptive trusted-class priority; an optional global bucket adds an identity-independent bound on untrusted admissions. The evaluation makes the resulting policy trade-off explicit: authenticated clients receive latency isolation, whereas tighter aggregate containment can reject legitimate unauthenticated offloads and reduce overall expected accuracy through edge fallback.

0.43

RetryRoute: Retry-Aware Model Routing for Agentic Image Editing Services

cs.LG ·2026-10-30 ·Hanshuai Cui, Zhiqing Tang, Zhi Yao, Jun Luo, Weijia Jia
hot direction similarity 0.58
Abstract

Web agents increasingly edit images by calling open editing models in a critique-and-retry loop, and these models differ in cost by more than an order of magnitude. After every rejected edit, a service must decide whether to retry the same model, escalate to another or give up. Existing routers pick one model per request, and cascades escalate in a fixed order. However, both treat a retry as an independent draw. We identify sticky failures, meaning that a retry after a failure succeeds only 17-53% as often as a first try across six open models, three benchmarks and two judges. Based on this finding, we propose RetryRoute, a retry-aware router that estimates each model's chance of success given the request's failure history. It plans retries, escalations and abandonment over failure states and adds live queue waits to the cost, so the same plan becomes load-aware dispatch. At the main budget, RetryRoute's success under the in-loop judge exceeds the strongest of eleven prior baselines by 2.8 points on average over eight groups. On a GPU serving testbed it raises the share of edits delivered within the service-level objective by 5.0-6.3 points over the best prior baseline at every load.

0.30

Cooldown Landmines: Cross-Tenant Interference Attacks on LLM Gateways

cs.CR ·2026-10-04 ·Yudong Gao, Linghan Chen, Wenhan Wu, Quan Shi, Xutao Mao +6 more
published similarity 0.58
Abstract · 24 pages, 7 figures. Main text in ICLR format; includes appendices

LLM gateways enforce separate tenant quotas while sharing model deployments and cooldown records that temporarily exclude failing backends. However, a tenant's request failure can update these shared records and restrict other tenants' access to serviceable deployments. We identify two attacks that exploit this gap in LiteLLM. The first uses requests rejected at the key's requests-per-minute (RPM) limit: caller-supplied identifiers for known registered deployments reach failure handling, allowing two rejected requests to redirect another tenant to fallback with zero upstream calls from those requests. The second uses admitted traffic to create cooldown records that persist after backend capacity recovers. To address these failures, we design an origin check that blocks deployment updates from key RPM rejections and tenant-scoped cooldown that preserves the triggering tenant's back-off while retaining shared records for backend faults. Experiments with authenticated proxies and self-hosted vLLM demonstrate that the attacks can force fallback or denial while deployments remain serviceable. Across five paired four-worker runs, tenant scoping reduces victim fallback after recovery from 54/60 to 0/60, while increasing attempts against exhausted shared quotas. These findings show that tenant isolation must cover both request admission and the failure handling that governs shared deployment availability.

0.39

SafeNom: Data-Aware Microservice Policies

cs.PL cs.NI ·2026-09-24 ·Karuna Grewal, P. Brighten Godfrey, Justin Hsu
similarity 0.58
Abstract · 27 pages, 5 figures

Many cloud-based applications are organized as loosely coupled microservices, where invoking a service's API triggers a cascade of APIs across many services and leads to inter-service exchange of API parameters and output responses. Current tools for monitoring microservice safety properties have limited expressiveness for properties that describe the flow of data through API calls. To this end, we present SafeNom, a specification and monitoring framework for microservices based on nominal languages. SafeNom policies can express both the desired order of API calls and how the data carried in requests and responses should or should not flow between the APIs. Policies are enforced using a nominal automaton-based distributed runtime monitor which can be applied in a blackbox and non-invasive manner, without access to the service implementation and without making changes to the service implementation. Our experiments show that our monitor can efficiently enforce rich data-aware properties while incurring minimal latency overhead, on the order of a few milliseconds.

0.32