CS Paper Radar

Procrustes-Bering-Sheaft: Automated Model Discovery and Virtual Crash Tests for Microservices

Anatoly A. Krasnovsky

cs.SE cs.DCcs.PF ·submitted 2026-10-22 15:40 UTC · v1

Abstract

Procrustes-Bering-Sheaft automates lightweight model discovery and virtual crash tests for microservices. Each test estimates operation availability by Monte Carlo sampling of service and communication failures. Procrustes checks deployment and tracing evidence; Bering discovers dependencies and incorporates replica declarations; Sheaft evaluates declared operation-success conditions. Independent command-line interfaces exchange reusable, versioned artifacts. The toolchain extends an earlier graph-based approach with mandatory-call contracts, failure-tolerance sweeps, baseline comparison, component-importance analysis, and CI/CD gates. We demonstrate the workflow on one HTTP operation. Six controlled execution states exposed two mismatches between reachability and observed success; an explicit required-call predicate matched all six. Two modelled one-replica additions produced different decisions: recommendation retained an availability-target violation, whereas catalog restored the baseline estimate of 0.979900 and grid tolerance of 0.12. Analytical oracles and controlled tests verify the implementation; archived replay checks numerical continuity with two earlier studies.

Comment
Submitted to the ICSE 2027 Tool Demonstration and Data Showcase track

Closest papers

Nearest neighbours by meaning across the whole index.

Measuring the Serving Stack Instead of the Model: Hidden Confounds in Local Tool-Use Evaluation

cs.CL cs.AIcs.SE ·2026-09-22 ·Lijuan Tang, Yuemeng Zheng
similarity 0.67
Abstract · 9 pages, 4 figures, 3 tables. Accepted at the 2nd Workshop for Research on Agent Language Models (REALM) @ EMNLP 2026

A coding agent must emit a valid tool call--a parseable invocation of a tool in the provided schema--before the harness can execute its chosen action. We study how local serving stacks affect this protocol step and show that measured outcomes can depend on the serving layer rather than model behavior alone. In Ollama, the default tools= request is gated per model by a static template flag: some models are accepted and return calls as text, some return native tool_calls, while Phi-3 and Gemma-3 are rejected before inference. In our harness, rejection and retry exhaustion are not preserved as structured failure metadata, so downstream analysis can misclassify them as model non-calls and naively report 0% fidelity. Adding a text tool list while retaining the native channel recovers much of the measured fidelity for accepted models, whereas a uniform text protocol reduces fidelity for Llama-3.2, which has native tool-call support. Cross-stack probes on Ollama, llama.cpp, vLLM, and SGLang show different handling of the same request. Constrained decoding removes parse failures but can induce non-termination, and turn-pooled versus per-instance estimates differ by up to about 55 points. We conclude with a checklist for treating serving behavior as part of the evaluation protocol.

0.55

RESTberus: Trace-Aware Benchmarking of Automated REST API Testing Tools Across Microservice Dependencies

cs.SE ·2026-11-03 ·John Toniutti, Massimiliano Baldo, Marino Miculan, Vincenzo Riccio
similarity 0.66
Abstract

Existing REST API testing benchmarks focus on isolated services, overlooking failures arising from interactions with downstream dependencies. In microservice systems, response bodies and entry-point operations alone cannot reliably distinguish failures, compromising tool comparisons. We present RESTberus, a Kubernetes-based benchmarking platform for black-box REST API testing tools on deployed microservice systems. By complementing API-level evidence with distributed traces, RESTberus can distinguish failures manifesting in the target service from those manifesting in downstream dependencies. Across six tools on two microservice benchmarks, our trace-aware deduplication strategy achieves 87.1% accuracy, outperforming baselines from prior work by 10-13 percentage points. Our empirical study shows that all tools expose downstream failures, accounting for 11.7% of estimated unique failures on average, which isolated-service benchmarks cannot expose. On single-service benchmarks, RESTberus produces statistically equivalent results to RESTgym, demonstrating that integration-level assessments can preserve compatibility with existing tools and comparability with prior evaluations.

0.27

Tool-Calling LLM Agents Under Realistic MCP Faults: Scale Does Not Purchase Fault Tolerance

cs.SE cs.AI ·2026-10-23 ·Antonio Ken Iannillo, Joshua Segun Owotogbe, Roberto Natella, Francesco Avallone, Kristina Kudryavtseva +1 more
publishedcode/data similarity 0.63
Abstract · Accepted at NeurIPS 2026 (poster). 10 pages of main text plus appendices and the NeurIPS checklist. Code and data: https://gitlab.com/unilux…

LLM agents increasingly rely on external tools exposed via the Model Context Protocol (MCP), yet current benchmarks assume tools respond correctly and on time, leaving agent resilience to realistic tool failures untested. We introduce MCP-Injector, a drop-in, protocol-aware fault-injection middleware that interposes on the MCP tool-call path with controlled persistence (permanent, transient, intermittent) and seedable schedules. Its fault model is calibrated to the observed incident distribution (failure mode x persistence) of the public MCP ecosystem, mined from 57,240 issue and pull-request threads across 1,774 open-source MCP repositories. In a 1,350-cell experiment crossing two backbone sizes (20B, 120B), three agent stacks (custom multi-round plan-and-execute, OpenAI Agents SDK, LangChain), and three fault injection conditions, we find that faults drawn from this permanent-heavy stress profile cause severe quality degradation (task-completion score falls by 60%, from 8.08 to 3.23 out of 10), and that scale does not purchase fault tolerance for the two gpt-oss backbones we test: the 120B retains no larger share of its fault-free score under faults (resilience-ratio model, $p = 0.96$), and its completion rate under faults is, if anything, lower (91.1% vs. 95.6%; difference 4.4 points, 95% CI $[-0.2, 9.3]$). Agent stacks differ in how well they absorb faults, which suggests that framework-level error handling matters more than model scale. These results demonstrate that capability benchmarks alone can mask operational brittleness, motivating protocol-aware fault injection as a complementary evaluation dimension for tool-calling agents.

0.55

Retry Amplification in Distributed Systems: A Systematic Analysis of Retry Policies and Their Role in Cascading Failures

cs.SE ·2026-08-26 ·Rishabh Mehan, Jasmit Kaur Saluja
publishedcode/data similarity 0.63
Abstract

Retry mechanisms are a standard component of resilient distributed systems, but their collective behavior, when every tier in a call path retries concurrently, is less well understood than the per-client guidance that produced them. This paper introduces the retry amplification factor (RAF), a metric quantifying the additional request volume that retry policies generate during partial failures. In a study of 200 open-source Python microservice projects, explicit retry logic is detected in 11.5%, and an audit of our own false negatives places true prevalence near 41%. Among the projects detected, 60.9% contain at least one configuration without backoff, and after manual verification exactly one of 113 production configurations randomizes its delay. We then evaluate these policies in simulation (n = 100 trials per strategy). Under correlated failures, a naive standard retry policy reduces the success rate from 55.4% to 41.5% relative to performing no retries at all. We catalog five recurring anti-patterns, propose Adaptive Retry Budgeting (ARB), and show that budget-constrained retries maintain success rates close to the no-retry baseline while still recovering from transient faults. These results indicate that retry behavior should be designed as a system-level property rather than configured locally at each call site.

0.66

A Resilient Runtime-Verification Fabric for Security Monitoring of Critical Edge-IoT Infrastructure

cs.CR cs.DC ·2026-10-05 ·Nikolaos Kekatos, Marinelio Chintri, Panagiotis Katsaros, Alexios Lekidis, Tom Nianios +3 more
publishedhighly cited similarity 0.62
Abstract · 21 pages, 2 figures. Accepted at the 21st International Conference on Critical Information Infrastructures Security (CRITIS 2026)

Protecting critical infrastructure increasingly depends on continuously verifying large IoT fleets against formal security specifications at runtime. Yet the runtime-verification (RV) pipelines proposed for this task are typically single-host prototypes whose monitors read a shared log file, with no resilience to the failures such deployments incur: a crash or overload silently drops events, clock skew corrupts the ordering metric monitors require, a time-triggered "node has gone silent" property cannot fire when the network itself falls silent, and one slow consumer stalls the pipeline. Each failure is silent: the monitor keeps emitting verdicts over a corrupted view. We present RV-Fabric, a resilient delivery layer that carries the hierarchy over two brokers (MQTT for device ingest, a durable stream broker for backend delivery) and re-establishes five continuity guarantees: durable delivery under crashes, a trusted event order, progress under total silence, consumer isolation and flow control under bounded overload, each an invariant conditioned on broker durability. Above the transport, RV-Fabric makes evidence completeness part of runtime-verification semantics: every verdict carries a status (sound, degraded, incomplete or unavailable) derived from delivery gaps, retention pressure and liveness, so an incomplete stream cannot yield an unqualified all-clear. Under controlled fault injection on a containerised testbed, measured against a fault-free oracle using the real MonPoly engine, the shared-log baseline misses six of seven injected incidents, reporting each as an unqualified all-clear, whereas RV-Fabric preserves all seven; removing a delivery mechanism reintroduces silent loss, removing isolation costs only timeliness. Two published critical-infrastructure datasets, water-SCADA and IoT/IIoT, replay end-to-end.

0.59

Prioritizing Microservice Requests via Retries

cs.NI cs.CRcs.DC ·2026-10-14 ·Ofek Lavi, Jhonatan Tavori, Hanoch Levy, Anat Bremler-Barr
similarity 0.61
Abstract

Cloud services rapidly shift towards independently-managed microservices architectures. This independence may lead to temporary microservices miscoordination, resulting in overloads and rejections of valuable transactions. This weakness may further be exploited by attackers to launch overload attacks inflicting performance and cost damages. Retry operations are a key machinery aiming at overcoming seldom rejections. However, during overload periods they: a) Fail protecting valuable transactions from rejections, and b) Trigger large-scale performance-degrading retry storms. We propose simple modifications in retry operations, overcoming these deficiencies by transforming retries into a rejection prioritization mechanism, DiffTry (Differential Retries). It is based on a simple, lightweight, distributed and stateless design and requires neither overload sensing nor microservices coordination. We develop an analytical model capturing DiffTry's effectiveness for two services in tandem, then generalize it to networks of services. Our theoretical framework lets practitioners quantify, for a given network, the resulting prioritization and rejection rates, and yields formal guarantees that DiffTry substantially reduces the impact of both incidental and adversarial orchestrated overloads. We validate these predictions in simulation and on a live Kubernetes deployment, where DiffTry is a simple drop-in service-mesh plugin requiring no application changes; the measured rejection closely matches the analysis.

0.18

What a Random Draw from the MCP Registry Contains, and What Tool-Use Benchmarks Contain Instead

cs.SE cs.AI ·2026-09-10 ·Haseeb Mohammed Afsar
benchmarkcode/data similarity 0.60
Abstract · 10 pages, 3 figures. Seeded, re-runnable pipeline and per-server outcomes: https://github.com/itguruhaseeb/mcp-probe ; archived at doi:10.52…

Studies of the Model Context Protocol (MCP) server ecosystem draw their samples in ways that quietly select for servers that work: reference sets, popularity lists, hand-curated frames, or pipelines that repair a server until it starts. We report what an unrepaired probability sample actually contains. From a 24,135-server registry census we draw 400 npm/stdio servers with a published seed and probe each one over the wire. Only 48.8% complete an initialize handshake, against 66.7% for a hand-curated frame measured with the same instrument, and the dominant failure is not missing credentials (13.3%) but servers that never start at all (37.5%). Among the 195 that do run, hard conformance is total: zero fatal JSON Schema violations across 2,766 advertised tools. Optional safety annotations are the real variance, and the tool-level omission rate on a random draw is 58.8% against 41.5% on the curated frame, so curation flatters this figure too. We then compare the tool descriptions these servers advertise against two tool-use benchmark corpora using one method held constant. Real MCP tools show 2.8% near-duplication at cosine 0.70, and all of it lies within single servers: cross-author near-duplication is 0.0% at every threshold tested. BFCL v4 shows 16.7%, of which 16.4 points lie between independently presented tasks. UltraTool shows 0.3%, cleaner than real tools, so this is a property of BFCL and not of synthetic corpora as a class. Separately, 68.8% of raw BFCL rows and 85.6% of raw UltraTool rows are exact name-plus-description repeats, against 0.4% for real MCP, so any statistic computed over these releases without global deduplication measures repetition rather than tools. All figures regenerate from released scripts and a published seed.

0.75

SilentProbe: Measuring Silent Failure in Production APIs Used as Agent Tools

cs.IR cs.SE ·2026-08-29 ·Zongrong Li, Shengkun Ye, Feiyou Guo, Zuoyou Dang
code/data similarity 0.59
Abstract · 12 pages, 9 figures. Code and data: https://github.com/Jasper0122/silentprobe

An LLM agent calling a production API cannot distinguish a query that matched nothing from a query the server did not understand. Both return HTTP 200 with a parsable body, no exception to catch and no field to branch on. We ask what predicts which one occurred, and what it does to the agent. Auditing 721,320 parameters across 2,501 independently published OpenAPI documents, we find that 7.5% declare an enumeration and 15.2% declare any machine-checkable constraint at all, while 40.1% of documents state at least one constraint in prose that their schema does not encode. Executing 219 schema-derived perturbations against live commercial endpoints from 27 vendors, reached through a single aggregation layer (Monid) that publishes a schema and returns a run identifier for every call, we find that constraint form, not vendor identity, predicts honesty: machine-checkable constraints yielded an honest error in 111 of 111 cases, prose-only constraints failed silently in 44 of 61 (p = 2e-13). Twelve models across eight families then met these endpoints on ordinary tasks. A vocabulary that the description merely exemplifies was missed by every model on 88 of 88 attempts, while vocabularies written out in full were used correctly 88 to 91% of the time. Running the full agent loop, models detected the resulting silent failure in 12% of cases, repaired it in 0%, asserted a false negative to the user in 41%, and invented a figure in 12%. Promoting the vocabulary into the schema removes the failure, from 88 of 88 to 0 of 89. The fix is one line of schema rather than a better model. Code, schemas, perturbation sets, agent transcripts and per-call run identifiers are released at https://github.com/Jasper0122/silentprobe.

0.60