CS Paper Radar

Polynomial Invariants for Probabilistic Transition Systems with Unbounded Support

Anne Schreuder, Lorenz Winkler, Laura Kovács, C. -H. Luke Ong

cs.LO ·submitted 2026-09-02 11:10 UTC · v1
publishedtheory

Abstract

We study the synthesis of polynomial invariants for probabilistic transition systems (PTS) based on martingale theory. We present tractable methods to verify that such polynomials are indeed invariants, in the sense that their expected value upon termination is the same as their value at the start of the computation. We do this by applying the Optional Stopping Theorem (OST) in the form of a specific precondition. This precondition requires the existence of an integrable dominating function for the martingale expression, which implies uniform integrability; we refer to this condition as dui. For linear PTS we simplify the dui property to proving finiteness of the expected value of an expression depending on the update matrix, the degree of the martingale expression, and the stopping time. Specifically, if all random samples have finite moments and we can verify a moment bound on the runtime of a linear loop, then we can automatically synthesise polynomial loop invariants that satisfy the OST. Notably, dui allows for the sampled distributions to have unbounded support, which is a novel contribution to the field.

Comment
16 pages + 19 pages appendix, extended version of submission accepted at LPAR'26

Closest papers

Nearest neighbours by meaning across the whole index.

Polynomial VASS MDPs

cs.LO ·2026-11-02 ·Michal Ajdarow
similarity 0.63
Abstract

Markov decision process over vector addition system with states (VASS MDP) is a finite state model combining non-deterministic and probabilistic behavior, augmented with non-negative integer counters that can be incremented or decremented during each state transition. VASS MDPs can serve as abstractions of probabilistic programs, offering many decidable properties. In this paper, we develop techniques for analyzing the polynomiality of asymptotic estimates for VASS MDPs. That is, for every initial configuration of size \(n\), we consider the number of transitions needed to reach a configuration with some counter negative. We show that for any VASS MDP, either for all \(ε>0 \) and all sufficiently large \(n\) the number of transitions is at least \(2^{n^{\frac{1}{2}-ε}} \) with probability at least \(1-ε\), or for any superpolynomial function \(f \), all \(ε>0 \) and all sufficiently large \(n\) the number of transitions is upper bounded by \(f(n) \) with probability at least \(1-ε\). Furthermore, if the VASS MDP is DAG-like then we can replace \(f(n)\) with \(n^{k+ε} \), where \(k=2\cdot 3^{w\cdot d+|T|} \) and \(w,d\) and \(T \) are the number of MECs, counters and transitions of the VASS MDP, respectively. It is decidable in polynomial time which case holds.

0.30

Quantitative coverability for probabilistic well-structured transition systems

cs.LO cs.FL ·2026-09-22 ·Raphaël Faure, Alain Finkel, Gaspard Fougea, Lina Ye
similarity 0.58
Abstract · 25 pages

Well-structured transition systems (WSTS) provide a classical framework for the verification of infinite-state systems, but their probabilistic extensions lack a unified treatment of quantitative coverability: path-enumeration algorithms assume a finite branching degree, while alternative approximation schemes defer some computations, such as probabilities over a bounded horizon, to the model at hand. We introduce probabilistic well-structured transition systems (pWSTS), Markov chains over countable state sets whose underlying transition systems are WSTS, with no a priori assumption on the branching degree. This class encompasses any WSTS equipped with a Markov kernel, such as probabilistic vector addition systems (pVAS) and probabilistic lossy channel systems (pLCS). For an effective subclass, we solve the approximate quantitative coverability problem over bounded horizons, and over infinite horizons under decisiveness, requiring no probabilistic information beyond individual transition probabilities. We then identify a general source of decisiveness: every stochastically monotone pWSTS is decisive with respect to every upward-closed set. We finally instantiate the framework on multi-type Galton--Watson processes, a classical model of population dynamics whose offspring distributions may have infinite support. Under mild assumptions on the reproduction laws, these processes are effective pWSTS, and they are stochastically monotone, hence decisive. Approximate quantitative coverability is therefore computable for them over both horizons, with a proof that uses none of the traditional tools: neither generating functions nor any case distinction between regimes.

0.41

A Hoare Logic for Continuous-Time Stochastic Processes

cs.LO cs.PLmath.PR ·2026-11-09 ·Kei Imada, Justin Hsu
similarity 0.57
Abstract

Programming language researchers have developed a wide array of tools to reason about probabilistic programs, but it remains unclear how to apply these proof techniques to continuous-time stochastic processes. To bridge this gap, we propose a verification method for properties of continuous-time stochastic processes by reasoning about them as standard programs. To demonstrate our approach, we define a continuous semantics for a probabilistic imperative language $\mathrm{pWhile}^{\mathrm{dt}}$ by taking the Kuratowski limit of the standard semantics, which collects limit points of the standard program semantics as the time-step approaches zero. Then, we present a Hoare-style logic that uses loop invariants and decreasing functions to verify properties of continuous programs. To establish soundness, we identify a broad class of assertions that are preserved from the standard semantics to the continuous semantics. To demonstrate our approach, we verify properties of ten continuous-time stochastic processes modeled in $\mathrm{pWhile}^{\mathrm{dt}}$, including variants of Brownian motion and a stochastic hybrid system.

0.41

Sufficient and Necessary Continuous Barrier-like Conditions for Discrete-Time Stochastic Reach-Avoid Verification

eess.SY ·2026-09-21 ·Bai Xue
similarity 0.57
Abstract

This paper develops necessary and sufficient barrier-like characterizations using continuous barrier functions for infinite-horizon reach-avoid verification of discrete-time stochastic systems. Existing results establish necessary and sufficient conditions in terms of functional inequalities involving measurable or lower semicontinuous barrier functions. However, the limited regularity of such functions may hinder their numerical approximation and computational synthesis. Building on our previous barrier-like condition for finite-horizon reach-avoid verification, we show that this condition can also be used for infinite-horizon reach-avoid verification and, under a uniform absolute continuity condition on the transition kernels, admits a continuous barrier function whenever the exact reach-avoid probability is strictly larger than the prescribed threshold for every state in the initial set. We further show that the resulting continuous barrier function can be uniformly approximated by a polynomial one while preserving the required barrier-like conditions. For polynomial systems, we formulate these conditions as polynomial positivity constraints over compact basic semialgebraic sets. Putinar's Positivstellensatz then converts the positivity conditions into sum-of-squares (SOS) certificates, yielding semidefinite programming (SDP) formulations for synthesizing polynomial barrier functions. We establish both soundness and completeness of the resulting SOS-based procedure. Finally, two numerical examples illustrate the theoretical results and demonstrate the resulting SDP approach.

0.34

Perturbation equivalence in labelled Markov chains

cs.LO cs.FL ·2026-09-11 ·Syyeda Zainab Fatmi, Stefan Kiefer, James C. A. Main, David Parker
theory similarity 0.56
Abstract

Behavioural equivalences, such as language equivalence and probabilistic bisimilarity, are fundamental techniques for reducing the size of probabilistic models. However, these equivalences are sensitive to the precise values of transition probabilities, making them unsuitable in applications where probabilities are subject to approximation. Motivated by settings in which the support graph of a labelled Markov chain is known but the transition probabilities are uncertain, we study robust variants of these equivalences. We introduce universal (perturbation) equivalence, which captures a variant of equivalence that is resilient to all perturbations of transition probabilities: two states or distributions are universally equivalent if they remain equivalent under every assignment of transition probabilities consistent with the support graph. We also consider the dual notion of existential (perturbation) equivalence, which holds whenever there exists an assignment of transition probabilities that yields equivalence. We establish that, for states, universal language equivalence coincides with universal probabilistic bisimilarity and develop a characterisation that yields a polynomial-time partition refinement algorithm. We implement the algorithm and demonstrate experimentally that it is effective as a technique for robust model reduction. We further show that universal language equivalence for distributions is closely related to the state case, and prove NL-completeness of deciding universal equivalence for both states and distributions. We prove that, for states, existential language equivalence coincides with existential probabilistic bisimilarity and deterministic witness transition functions always suffice, leading to an NP-completeness result. In contrast, we show that existential language equivalence for distributions is complete for the existential theory of the reals.

0.32

Certifying Almost-Sure Reachability in Unknown MDPs

cs.LO cs.AI ·2026-10-13 ·S Akshay, Suguman Bansal, Gaurav Mahla
theory similarity 0.56
Abstract

{\em Specification-Guided Reinforcement Learning (RL)} has largely concerned itself with \emph{learning} an optimal policy against a formal specification in an unknown {\em Markov Decision Process (MDP)}. We take up the dual question. Given a policy already in hand, can one \emph{verify} that it is correct against a formal specification in an unknown MDP, using sampling access alone? Such a question arises naturally in safety-critical systems. We investigate it for the {\em almost-sure reachability property}, whose correctness is witnessed by a {\em ranking supermartingale (RSM)} certificate. Our first finding is negative. In the absence of any further information about the MDP, one can neither learn such a certificate nor verify a candidate one with probabilistic guarantees. On the positive side, we show that, under a mild restriction, a known lower bound $p_{\min}$ on nonzero transition probabilities, a correct certificate \emph{can} be learned with PAC-style guarantees. To the best of our knowledge, this is the first study of learning and verifying certificates for almost-sure reachability in unknown MDPs from sampling access alone. Finally, our experiments show that the $p_{\min}$-based sufficient condition, while sound, is conservative: the learned certificate often remains valid on the true MDP even when the condition fails.

0.07

Poisson Tangent Limits and Critical Policy Switching for Sampled Bellman Operators

eess.SY ·2026-08-12 ·Ming-Zhe Dai, Chengxi Zhang
similarity 0.55
Abstract

Consider a discounted Markov decision process with continuous action space in which, at each state visit, the controller draws a random pool of $N$ candidate actions and selects among them. When the optimal action set has zero mass under the sampling distribution, the value of this random-candidate model converges to the optimal value as $N$ grows, but the rate of convergence and the asymptotic selection rule are governed jointly by the geometry of the optimal set and by the transition kernels of the near-optimal candidates. This paper develops an exact first-order theory of both. The rescaled near-optimal candidates converge to a marked Poisson point process, and the leading term of the value gap is the fixed point of a nonlinear tangential Bellman operator, a stochastic generalization of the classical resolvent that emerges when several optimal actions compete. The asymptotic selection at exact ties is genuinely dynamic, driven by the transition kernels through the fixed point, and admits an explicit Mecke integral formula at the limiting Poisson level; perturbing the tie at the critical rate yields a switching layer in which the value gap and the selection interpolate continuously between the branch regimes, and in the unique-optimum heterogeneous case the critical class is propagated at the slowest global scale through the discounted reachability of the limit policy. Numerical experiments confirm the predicted rates, constants, and selection probabilities.

0.04

Probabilistic Simple Types for Almost-Sure Termination (Extended Version)

cs.LO ·2026-11-17 ·Ugo Dal Lago, Willem Heijltjes
similarity 0.54
Abstract · Extended version of the published paper at CSL 2027

We present a type system for almost-sure termination (AST) in higher-order probabilistic computation. Our vehicle is the recent Functional Machine Calculus (FMC), which combines the lambda-calculus with effectful imperative computation. Importantly for our purpose, it models branching and looping computation with sum types and a typed iterator construct. Our key observation is that by replacing sum types with probability distributions, types ensure that iteration has a non-zero chance of exiting, which then gives AST in a simple and natural way. Concretely, our type system assigns explicit probability distributions to termination outcomes, allowing probabilistic behaviour to be tracked compositionally at the type level. Types then soundly guarantee almost-sure termination, even in the presence of higher-order functions. The fragment of ground terms, which consists purely of the branching and iteration constructs with probabilistic sampling, is equivalent to finite Markov chains and is positively AST (expected evaluation time is finite). The full calculus is more expressive, which we demonstrate through an example term with infinite expected evaluation time.

0.34