CS Paper Radar

Reference-Dependent Utility Optimization for Goal-Oriented Communication Systems

Symeon Vaidanis, Photios A. Stavrou, Marios Kountouris

cs.IT cs.NIeess.SP ·submitted 2026-11-09 14:29 UTC · v1

Abstract

Traditional communication resource allocation optimizes operational metrics such as rate, latency, energy efficiency, and reliability. Motivated by goal-oriented semantic communication, we develop a generalized allocation framework that transforms operational rewards through reference-dependent user utilities and evaluates them via expected utility, cumulative prospect theory (CPT), or related decision criteria. The formulation separates operational rewards, user-side valuations, and system-level decision criteria under uncertainty. We characterize utilities incorporating QoS/service reference levels, deficit sensitivity, post-reference saturation, and loss aversion. Weighted sum-reward maximization, proportional fairness, $α$-fairness, and stochastic expected utility maximization emerge as exact special cases, while soft QoS-aware allocation follows from a deficit-penalty generalization. For deterministic orthogonal systems, we derive utility-weighted marginal-reward KKT conditions characterizing global optimality under concave composite objectives. For nonorthogonal and stochastic systems, we develop successive convex approximation (SCA) algorithms and establish stationary-point guarantees under explicit smooth and composite nonsmooth assumptions, including a CPT branch-stationarity result under eventual nondegenerate ordering identification. For globally solved penalized problems, we derive an $O(1/λ)$ bound on excess aggregate soft-QoS deficit, and on the deficit itself whenever a QoS-feasible allocation exists. Additionally, we establish an $O(1/ε^2)$ outer iteration bound for the smooth fixed-step SCA best-response residual. Numerical examples demonstrate how reference points, marginal utility slopes, channel quality, and resource budgets influence active users and allocation policies.

Comment
This paper has been submitted for publication

Closest papers

Nearest neighbours by meaning across the whole index.

QoE-Driven Hybrid Resource Allocation for Multi-User OFDMA Semantic Communications

eess.SP ·2026-10-23 ·Xiaotian Xu, Zhifeng Tang, Guangchen Wang, Nan Yang, Xiangyun Zhou +2 more
similarity 0.62
Abstract · 6 pages, 8 figures

In this paper, we develop a quality-of-experience (QoE)-driven hybrid resource allocation framework for multi-user downlink orthogonal frequency-division multiple access semantic communication systems, where the hybrid framework jointly handles continuous and discrete decision variables. Specifically, we formulate a mixed continuous-discrete optimization problem to jointly optimize transmit power, subcarrier allocation, and semantic decisions under coupled power and spectrum constraints. To better characterize semantic service quality, we propose a preference-inverse multiplicative exponential QoE that is bounded and non-compensatory, capturing the intrinsic ``both-required'' property of semantic rate and similarity. To solve the optimization problem, we develop a projection-based multi-agent deep deterministic policy gradient framework that enforces feasibility for continuous and discrete actions. Furthermore, we integrate Gumbel-softmax relaxation with an adaptive performance-driven temperature annealing scheme to enhance the learning of discrete semantic decisions. Simulation results demonstrate that our proposed framework achieves superior convergence, greater stability, and higher QoE compared to the conventional QoE design and penalty-based or fixed-temperature baselines, highlighting the benefits of our QoE design and the effectiveness of hybrid decision learning in semantic communications.

0.30

Proximal Preference Optimization: A Unified Framework for Stable and Reward-Free Alignment

cs.LG cs.AI ·2026-11-07 ·Qisai Liu, Zhanhong Jiang, Zahid Hasan, Soumik Sarkar
theorycode/data similarity 0.57
Abstract · 10 pages, 4 figures, IEEE BigData 2026

We introduce a proximal preference optimization framework that unifies the implicit reward formulation of Direct Preference Optimization (DPO) with the trust-region stability of Proximal Policy Optimization (PPO). Our framework yields two complementary algorithms. P$^2$O (Proximal Preference Optimization) introduces a clipped importance ratio directly into the DPO preference margin, producing a reward-model-free algorithm that supports safe dataset reuse, bounded policy divergence, and monotonic preference improvement guarantees. PKTO (Proximal Kahneman-Tversky Optimization) extends P$^2$O by replacing the DPO margin with KTO's prospect-theoretic utility margins, adding a self-regulating adaptive baseline that tightens the effective trust region as the policy drifts, and an asymmetric weighting that handles imbalanced or asymmetric preference datasets. We provide a unified theoretical analysis, including Kullback Leibler (KL) trust-region bounds, a policy-improvement lower bound, and convergence guarantees under multi-epoch proximal updates. Empirically, P$^2$O and PKTO match or exceed DPO and KTO in alignment quality while maintaining better stability under repeated dataset reuse across multiple LLMs on three preference benchmarks. The code base is available here.

0.33

FARM: Fundamental Agentic Reward Model For Multi-task Wireless Network Optimization

eess.SY ·2026-10-02 ·Feiran You, Changxu Ni, Haozhe Ma, Jun Li, Hongyang Du
similarity 0.55
Abstract

Future wireless networks require learning agents to adapt across heterogeneous channel conditions, traffic patterns, quality-of-service (QoS) requirements, objectives, and operational constraints. Reusing decision knowledge across such tasks is challenging because conventional multi-task and transfer reinforcement learning methods primarily share or transfer policies, coupling transferable knowledge with task-dependent action mappings. This paper proposes FARM (Fundamental Agentic Reward Model for Multi-task Wireless Network Optimization), a reward-space transfer framework that shifts cross-task knowledge reuse from policy space to trajectory-level decision evaluation. FARM introduces an Agentic Reward Model (ARM) that learns a task-conditioned reward prior from heterogeneous source-task trajectories and provides auxiliary guidance for task-specific policy optimization. In Stage I, ARM jointly models task conditions, temporal trajectory dependencies, and objective-dependent reward structures while each source task retains its own controller. In Stage II, the learned reward prior is frozen and reused to guide the adaptation of a target-specific controller for previously unseen tasks, without transferring source-task policies. Experiments on heterogeneous multi-access edge computing (MEC) tasks show that FARM achieves a mean late-stage gain of 29.8% over Single-task SAC on unseen Rate-Latency targets, compared with 16.1% for CRA Transfer, and reaches a 46.2% gain on the moderate-OOD FAR-M case. Further analysis shows that both Mamba and Transformer trajectory encoders support Reward-Space Transfer, while Mamba provides improved robustness as longer history dependencies are introduced.

0.18

Dual-Based Weight Selection for Approximate Linear Programming

eess.SY math.OC ·2026-08-25 ·Su Li, Andre A. Cire, Adam Diamant, Vahid Sarhangian
similarity 0.55
Abstract · 20 pages, 9 figures

Approximate Linear Programming (ALP) is widely used for large-scale Markov Decision Processes (MDPs), but its performance can be sensitive to the choice of state-relevance weights, which are typically selected heuristically. Performance bounds suggest aligning these weights with the discounted occupancy measure of the induced policy, and existing primal approaches address this through repeated greedy-policy construction. Nonetheless, they lack convergence guarantees and are computationally expensive. We propose a dual-based method that uses projected occupancy information from the ALP dual solution to construct a smooth stochastic policy and update the state-relevance weights, which avoids separate greedy-action calculations. We establish conditions under which the weights match the discounted occupancy of the induced policy and prove uniqueness and global convergence under appropriate smoothing. We also derive an a posteriori policy-loss bound that separates error from the weighted Bellman residual, occupancy mismatch, and stochastic-versus-greedy disagreement. Experiments on classical queueing and multi-priority scheduling problems show that the proposed approach reduces sensitivity to fixed weights and achieves comparable or better policy quality than primal updates at lower computational cost. Finally, we show that adaptive weighting is most valuable when the basis functions are sufficiently expressive for occupancy information to influence the resulting policy.

0.28

Queue-Aware ISAC Power Allocation Under Imperfect CSI: Integer-Service Analysis and Anchored Lyapunov Look-Ahead

cs.NI ·2026-10-18 ·Hanyi Sun, Yifan Zhou, Yixiao Gu
similarity 0.55
Abstract

We study power allocation for a multiuser integrated sensing and communication (ISAC) downlink with bursty queues and imperfect channel state information (CSI). Starting from a long-run utility--backlog objective, we derive its common task reward and online score. Integer departures are represented by threshold events of indefinite Gaussian quadratic forms, accounting for estimation error in both signal and interference. Their moments give the conditional one-slot task reward and characterize the error of an efficient plug-in score. An anchored quadratic Lyapunov look-ahead corrects its reward-greedy action with a bounded nominal loss. Across five CSI-error levels, the rule reduces mean queue exceedance by 27.4--43.9\% (0.49--0.99 percentage points) relative to the plug-in maximizer, with mean reward loss below 0.01 per slot. The rule achieves higher reward and lower exceedance than DPP, SAC and PPO; its advantage over freshly retrained learning baselines extends to five configurations with 9--23 power-allocation variables using fixed candidate budgets. For the seven-variable system, a fused implementation reproduces all 25 confirmation trajectories and achieves a one-core p99 decision time of 0.274--0.311\,ms.

0.21

Resource-Constrained Semantic-Aware Remote Estimation with Overlapping Sensor Coverage

cs.IT ·2026-09-07 ·Bowen Sun, Nikolaos Pappas
published similarity 0.54
Abstract

We study semantic-aware remote estimation of multiple finite-state Markov sources observed by K sensors with overlapping coverage. The sensors share a time-division multiple-access uplink and differ in transmission reliability, delivery delay, and transmission budget. In each slot, the scheduler jointly selects a source and one of its monitoring sensors, or remains idle, to minimize the long-run average cost of actuation error subject to global and per-sensor transmission-frequency constraints. We formulate this problem as a finite average-cost constrained Markov decision process. We show that the transmission resource functions have rank at most K, although the global constraint may still restrict the feasible region. Consequently, the Lagrangian depends only on K effective transmission costs, and an optimal constrained solution can be represented using at most K+1 deterministic policy-recurrent-class components. We further characterize the piecewise-affine concave Lagrangian value function and derive projected dual subgradient ascent over an explicitly bounded multiplier set. Numerical results illustrate the value-function structure, the need for policy randomization in a representative instance, and the interaction between global and per-sensor transmission budgets.

0.25

KKT-Perturbation-Aware Communication for Fully Distributed Online Convex Optimization with Nonseparable Coupling

cs.LG ·2026-11-09 ·Zhaoye Pan, Tao Li, Yunjie Guo, Guanpu Chen
similarity 0.54
Abstract

Nonseparable costs and coupled constraints in distributed online convex optimization often arise when system performance and operational constraints depend on the joint configuration of all agents and cannot be decomposed into individual contributions. Such nonseparability leads to fully distributed decision-making with challenging partial local information, where each agent evaluates its local primal-dual update using estimated global information via a local belief of the joint action and the shared Lagrange multiplier to be exchanged with neighbors. While the belief consensus established by prior works ensures no-regret learning, exchanging beliefs comprising full-dimensional primal and dual states at every round imposes a communication overhead that scales with both the joint decision dimension and the network size. Existing event-triggered schemes typically activate communication according to state variation or consensus error. Under nonseparable coupling, however, errors of equal magnitude can produce markedly different perturbations after passing through the KKT operator. We therefore propose a randomized KKT-perturbation-aware communication mechanism, where the triggering of each agent's primal and dual communication channels depends on their impact on the KKT operator. The proposed mechanism transfers the adaptive-sampling principle of coordinate descent from allocating computation across decision blocks to allocating communication across information channels: channels with larger estimated perturbations receive greater priority for refresh. Under standard regularity assumptions, the resulting algorithm achieves $O({T}^{2/3})$ expected static regret and $O({T}^{2/3})$ expected cumulative constraint violation. Numerical experiments demonstrate improved learning and constraint-enforcement performance per transmitted bit over belief consensus baselines.

0.19

Online Resource Allocation with an Endogenous Markov State: Fewer LP Solves Earn More

cs.LG ·2026-10-07 ·Zhaohua Chen
benchmark similarity 0.54
Abstract

We study finite-horizon online resource allocation with i.i.d. requests and an endogenous Markov state on a finite state space: each action affects the transition of the state that governs future rewards and resource consumption. In this problem, a transient fluid LP benchmark upper bounds the expected reward of every nonanticipating policy, while a stationary LP supplies randomized state-dependent controls. We assume that the stationary LP has a unique optimum and identify primal nondegeneracy and irreducibility of the optimal induced kernel as important regularity conditions in this framework. With a known request prior, we show that, under nondegeneracy and irreducibility, both frequent and infrequent re-solving attain $O(1)$ regret. However, under a degenerate optimum, irreducibility yields the sharp worst-case $Θ(\sqrt{T})$ rate for infrequent re-solving, while frequent re-solving can incur $Ω(T)$ regret. Thus, more frequent optimization can perform asymptotically worse. With an unknown request prior, we develop a three-phase U-shaped infrequent re-solving policy that coordinates learning and inventory correction with $O(\log\log T)$ LP solves. When the optimal induced kernel is irreducible and the algorithm is given the optimal target state class and a constant-cost entrance policy, it attains $O(1)$ regret under nondegeneracy and $O(\sqrt{T})$ regret under degeneracy. Without the target-class information, linear minimax regret is unavoidable. Numerical experiments further illustrate the instability of round-by-round re-solving relative to epoch-wise infrequent re-solving, show that thresholding greatly mitigates its loss, and find that infrequent schemes remain dominant under both known and estimated priors.

0.19