CS Paper Radar

Online Scheduling for Throughput Maximization of Time-varying Markovian Channels with Unknown Statistics

Tasmeen Zaman Ornee, Clement Kam, Ness B. Shroff

cs.NI ·submitted 2026-08-19 19:31 UTC · v1

Abstract

We consider a wireless scheduling problem in downlink wireless networks with unknown channel statistics, where a Base Station (BS) sends data to multiple users. The scheduling performance relies heavily on accurate Channel State Information (CSI), which is often costly to acquire. In this paper, CSI is obtained from ACK/NACK feedback, only after each scheduled transmission. Due to limited wireless channel resources, all users cannot be scheduled for transmission simultaneously. Hence, the most recently observed CSI can be outdated. The traditional approach to solve scheduling problems using outdated CSI is to utilize belief states, which are calculated using the time correlation statistics of channels. However, channel statistics are often unknown; consequently, belief states can be uncountable and this approach becomes infeasible. In this paper, we introduce a new sufficient statistics for the wireless scheduling problem. Towards this effort, we characterize the CSI staleness by the Age of Channel State Information (AoCSI) and show that the latest observed CSI and its AoCSI is a sufficient statistic of the history to make the scheduling decisions. Accordingly, we are able to reduce the state space for online learning. Our goal is to develop an online scheduling algorithm that maximizes the expected sum throughput of all users over a finite time-horizon while satisfying a channel resource constraint. The formulated problem is a Restless Multi-armed Bandit (RMAB). We develop an online Maximum Gain First (Online-MGF) policy, which achieves sub-linear regret on the number of episodes. For a special case of ON/OFF channels, we are able to prove indexability and derive a closed-form expression of the Whittle index. Numerical results demonstrate that the Online-MGF policy converges to MGF and Whittle index policies with known statistics within a very few episodes.

Comment
19 pages, 8 figures. This manuscript is currently under review in IEEE/ACM Transactions on Networking

Closest papers

Nearest neighbours by meaning across the whole index.

Joint Freshness and Age-Dispersion Control over Finite-State Markov Wireless Channels

eess.SY cs.NIcs.PF ·2026-09-15 ·Aresh Dadlani, Hina Tabassum, Muthukrishnan Senthil Kumar, Masoumeh Moradian
similarity 0.64
Abstract · 6 pages, 5 figures

Age of information (AoI) has become a standard design objective for timely monitoring as it measures the freshness of the latest update at a receiver. AoI alone, however, is insufficient in goal-oriented applications where decisions depend on consecutive observations. Age dispersion complements AoI by measuring the generation-time separation between consecutive updates. In this paper, we study joint freshness and dispersion control for a generate-at-will status update link modeled as a finite-state Markov wireless channel. The objective is to minimize the long-run probability that either AoI or age dispersion exceeds a prescribed threshold under an average transmission rate constraint. For channel-dependent randomized transmission policies, we derive exact matrix expressions for the stationary joint distribution of AoI and dispersion. For reversible channels, mean dispersion is lower bounded by the reciprocal of the delivery throughput, and the difference is an explicit non-negative variance term for which we establish the exact equality condition. We then formulate the adaptive joint-threshold control problem as a constrained Markov decision process and prove that it admits an exact finite-state representation whose size scales linearly with the number of channel states and the threshold levels. Simulation results show that the controller reduces joint threshold violations by 37.3% relative to adaptive AoI-only control under the same transmission budget, while increasing mean AoI by 8.3%.

0.15

Analysis and Optimization of Age of Information in Large-Scale Random Access Networks

cs.NI ·2026-10-20 ·Yueqi Chen, Siqi Fan, Zhuoyi Zhao, I-Hong Hou, Igor Kadota
similarity 0.62
Abstract · 8 pages, 1 figure, 2 tables

We consider a large-scale network where multiple nodes transmit time-sensitive information via a shared channel to a wireless base station using a Carrier-Sense Multiple Access (CSMA) protocol. In this network, packet transmissions from different nodes are interdependent, making it challenging to analyze and optimize information freshness, captured by the Age of Information (AoI) metric. In this paper, we develop a tractable analytical framework for analyzing and optimizing the AoI. By combining mean-field approximation with first return time and renewal reward analysis, the proposed approach provides a fast, tractable, and accurate AoI approximation that explicitly accounts for carrier sensing, enabling us to quickly find the network parameters that minimize AoI. Since the framework is built on a finite-state Markov representation of the CSMA protocol, it applies to a broad class of Random Access systems. As a concrete example, we study age-threshold CSMA protocols and derive closed-form or efficiently computable AoI approximations under different protocol settings. We validate the proposed AoI analysis via extensive simulations, demonstrating a tight agreement between analysis and simulation.

0.28

Age of Incorrect Information for Pull-Based State Estimation of General Markov Sources

cs.IT cs.NI ·2026-08-13 ·Marco Zanni, Mohamad Assaad, Touraj Soleymani
similarity 0.61
Abstract

We study pull-based remote state estimation of an arbitrary, multi-state Markov source while accounting for both freshness and correctness attributes of information. To that end, we formulate a discounted optimization problem in terms of the age of incorrect information (AoII), and express it as a joint source-AoII belief Markov decision process (MDP) under maximum a posteriori (MAP) estimation. We then exploit the information structure of the model and prove that every reachable belief is represented by the last successfully observed source state and the number of time slots elapsed since that observation. For numerical computation, we truncate the elapsed no-success duration at a finite level and derive an explicit error bound and a criterion for selecting the truncation parameter. For reliable links, we show that an optimal policy can be represented by a look-up table of waiting times. For unreliable links, we propose a persistent policy and derive computable performance bounds. We also show that the MAP estimate stabilizes after a finite number of time slots. To further reduce memory requirements, we introduce a hybrid estimator with an early stationary switch and derive a computable bound on the resulting difference in performance. Finally, we extend the framework to multiple sources, formulate the scheduling problem as a restless multi-armed bandit, establish a sufficient condition for indexability, and develop an approximate Whittle index policy based on interpolation. Our numerical results illustrate the structure of the optimal single-source policy, evaluate the performance of the multi-source policies, and verify that the proposed heuristic policies closely approach the optimal solution while substantially reducing computational efforts.

0.04

Minimizing Age of Incorrect Information in Pull-Based Remote Estimation of a Markov Source

cs.IT cs.NImath.PR ·2026-10-15 ·Ismail Cosandal, Sennur Ulukus, Nail Akar
similarity 0.59
Abstract

In this paper, we consider a discrete-time pull-based remote estimation system involving an information source and a remote monitor where the information source is a finite-state irreducible discrete-time Markov chain (DTMC) process. In this system, the remote monitor sends a pull request towards the source which responds by sampling its process so as to transmit the sampled values towards the monitor through a delay channel, encapsulated in information packets. We consider i) zero delay (ZD) and ii) unit delay (UD) channels, where the packet transmission times are zero and one slot, respectively. For remote estimation, we use the maximum a posteriori (MAP) estimator, and for quantifying its performance, we use the mean age of incorrect information (MAoII) which is a recently proposed freshness and mismatch metric that penalizes an incorrect estimation along with its duration. Our goal is to obtain pull policies so as to minimize MAoII under a long-run sampling rate constraint. For the ZD model, we propose a semi-Markov decision process (SMDP) formulation of the problem, which allows us to find the optimum observation-dependent pull policy. On the other hand, the UD channel prevents the monitor from possessing perfect information on the actual AoII process at any given time, and the SMDP formulation of the ZD channel cannot readily be used for the UD model. For the system with the UD channel, we first provide a numerical algorithm to calculate the MAoII and the average sampling rate for a given observation-dependent pull policy. For the optimization of the underlying remote estimation system, we use the joint distribution of the age and source processes to be obtained from the history of all observations, termed as belief. Subsequently, we propose a belief-dependent policy obtained via the belief-MDP formulation and a constrained deep reinforcement learning (CDRL) algorithm.

0.11

Survival-Time Risk Beyond Low-Order Channel Models: Identifiability and Scheduling

cs.NI cs.IT ·2026-11-02 ·Ruifeng Zheng, Zhihan Xu, Pit Hofmann, Frank H. P. Fitzek
similarity 0.58
Abstract · 6pages, 4 figures

This paper investigates whether low-order packet-loss statistics suffice to assess survival-time risk under bursty packet losses, i.e., the probability that no update succeeds for a prescribed number of consecutive slots. Conventional models, such as the Gilbert model, characterize packet losses using the loss rate and mean burst length, but channels sharing these statistics may exhibit substantially different failure risks. We characterize the Markov channels one order above a given low-order fit that share its statistics and analyze their risks under acknowledgment-based scheduling with transmission budgets. We show that, with dense transmission attempts, risk is primarily governed by loss persistence beyond the modeled memory, and the worst-case channel need not be the most persistent one. Under limited budgets, retry spacing makes loss relapse probabilities important and can reverse the risk ordering of compatible channels. Numerical and measurement results show that low-order models can substantially underestimate the risk, highlighting the need for scheduling-dependent channel statistics.

0.28

Fundamental Tradeoffs in Sampling, Transmission and Information Freshness: Instantaneous Feedback

cs.IT cs.NIeess.SY ·2026-11-16 ·Subhankar Banerjee, Sennur Ulukus
theory similarity 0.58
Abstract

We study a time-slotted generate-at-will status-update system over an erasure channel with transmission success probability $q$, subject to average sampling- and transmission-rate constraints $f_s$ and $f_r$, respectively. The sampler-transmitter pair jointly decides whether to idle, retransmit a stale packet, or replace it with a fresh sample. We derive a universal lower bound on the optimal average age of information, analyze a class of stationary randomized policies, and obtain an optimal policy within this class that provides a $4$-approximation guarantee. The joint problem reduces to the sampling-constrained problem when $f_s\leq qf_r$ and the transmission-constrained problem when $f_s\geq f_r$. For $qf_r<f_s<f_r$, we formulate a constrained Markov decision process (CMDP) and introduce the class $\barΠ$ of $(W,R)$ policies, where $W$ specifies a waiting threshold for transmission and $R$ specifies a packet-replacement threshold. We prove that, for every choice of Lagrange multipliers, the corresponding Lagrangian-relaxed Markov decision process admits an optimal policy in $\barΠ$. We then formulate a linear program (LP) over distributions of $(W,R)$ policies and prove that choosing a policy according to an optimal LP solution yields an optimal CMDP policy. Moreover, there exists an optimal LP solution assigning positive probability to at most three $(W,R)$ policies. Since $\barΠ$ is countably infinite, for every $ε>0$ we construct a finite subset $\barΠ_ε\subset\barΠ$ and show that the LP restricted to this set yields an $ε$-optimal policy. Finally, we derive a stronger pair of lower bounds by considering systems with only the sampling-rate or transmission-rate constraint. Using these bounds, we construct a policy randomizing among four $(W,R)$ policies with a $2$-approximation guarantee for $qf_r<f_s<f_r$.

0.12

Semantic Freshness Optimal Sampling and Transmission for Gossiping Receivers

cs.IT cs.NIeess.SP ·2026-08-31 ·Irtiza Hasan, Ahmed Arafa
publishedtheory similarity 0.56
Abstract · To appear in IEEE ITW 2026

We study the optimal joint sampling and transmission policy for a transmitter communicating with two gossiping receivers that share information with each other, with the objective of tracking a source under the Version Age of Information (VAoI) metric. The transmitter can observe source-version changes, but it has to pay a sampling cost to get the current source information content. Similarly, it can communicate with a receiver by paying a transmission cost. Gossiping enables local information exchange and is able to reduce costly direct transmissions. With imperfect communication links, we formulate an infinite-horizon average-cost Markov Decision Process (MDP) to jointly minimize receiver VAoI, sampling cost, and transmission cost. Using Relative Value Iteration (RVI), we evaluate the optimal policy and establish several properties of its structure. We prove that sampling has a threshold structure in the transmitter VAoI. Among direct transmissions, it is optimal to serve the older receiver. We further characterize the transmit or idle decision through the receiver VAoI difference. Our analysis shows that link reliability and receiver VAoI imbalance have a significant effect on the optimal policy structure. Numerical results verify the structural properties and demonstrate the performance gains of the optimal policy over multiple baselines.

0.47

Contention Resolution with a Fast-but-Fickle Channel

cs.DC ·2026-11-09 ·Umesh Biswas, Vini Chaudhary, Shafqat Hasan, Maxwell Young
theory similarity 0.56
Abstract

The contention-resolution problem studies how $n$ devices share a communication channel without centralized coordination. Classical models typically assume a single continuously available channel and focus on minimizing total completion time. However, modern wireless systems increasingly rely on shared spectrum, where access may be abruptly revoked by incumbent users. Motivated by this setting, we study contention resolution with two channels: a reliable slow channel and a faster channel that can be intermittently disabled by unpredictable eviction periods. Moving a batch of packets from the slow channel to the fast channel incurs a unit cost, while evictions from the fast channel impose a larger per-packet penalty. Our goal is to achieve asymptotically optimal latency while minimizing switching overhead relative to an optimal offline algorithm that knows the eviction schedule. We present Randomized Adaptive Channel Exploitation (RACE), a randomized algorithm that exploits the fast channel when available while maintaining progress on the slow channel during disruptions. When the fast channel has rate $ρ\in \bigl[Ω(\log n), O(\sqrt n)\bigr]$ and a sufficiently large constant-factor resource augmentation, RACE achieves an $O(1)$ competitive ratio for expected latency. RACE's expected switching-cost competitive ratio relative to any optimal-latency offline algorithm OPT is $O(ρ+\sqrt{n/m})$, where $m \geq 1$ is the number of fast-channel availability intervals used by OPT. We also prove a worst-case $Ω(\sqrt n)$ switching-cost competitive ratio for latency-competitive randomized algorithms, and a separate $Ω(ρ)$ switching-cost competitive-ratio lower bound. These lower bounds hold even with fixed constant-factor augmentation on both channels.

0.31