CS Paper Radar

Priority-Aware Deficit Round Robin for AXI4 Bandwidth Fairness in ASAP-7nm Process

Tasmin Khan, Tishna Sabrina, Ch. Md. Rakin Haider

cs.AR ·submitted 2026-11-08 06:15 UTC · v1
benchmark

Abstract

Modern systems-on-chip place real-time clients, processor cores and bulk data engines on the same AXI4 interconnect, where they contend for shared slaves. AXI4 fixes the ordering rules and carries a 4-bit quality of service (QoS) field, but leaves the service policy to the implementer, and that policy determines how long each master waits and how much bandwidth it receives once a slave saturates. We propose a QoS-aware deficit round robin (DRR) architecture that charges each grant in beats, and benchmark it against two architectures built behind the same port list: round robin, and QoS priority with anti-starvation aging. Charging in beats recovers bandwidth fairness while leaving every burst whole, so the address phase is issued once and locality for memory access is preserved. All three are driven by the same deterministic traffic generator and taken from RTL to GDSII on the ASAP7 7-nm predictive PDK. Under a 1:16 burst-length mismatch, the round robin and QoS architectures both hold the beat ratio between the two competing masters at 1:16, while our design brings it to 1:1 at full load. Mean read latency for the priority master falls from 95 cycles under round robin to 67 with QoS and 37 with QoS-aware DRR. At 500 MHz this serves the priority master in 74 ns against 190 ns. The measurements give a per-policy price and a traffic-based guide for choosing among the three.

Comment
6 pages, 5 figures

Closest papers

Nearest neighbours by meaning across the whole index.

DualPI2 Active Queue Management in ns-3: Implementation And Validation

cs.NI ·2026-08-12 ·Maria Eduarda Veras, Eduardo Freitas, Assis T. de Oliveira Filho, Djamel Sadok, Judith Kelner
similarity 0.69
Abstract

The demand for ultra-low latency applications necessitates advanced network architectures like the Low Latency, Low Loss, and Scalable Throughput (L4S) standard. A core component of L4S is the DualPI2 Active Queue Management (AQM), which ensures the safe coexistence of scalable and classic traffic. Despite L4S's growing adoption, the ns-3 network simulator lacks a high-fidelity, up-to-date DualPI2 model. This paper presents a comprehensive implementation of the DualPI2 AQM in ns-3, while also mirroring the official Linux Kernel architecture. Our model incorporates representative mechanisms previously absent in simulation, such as credit-based Weighted Round Robin (WRR) scheduling, step-marking, and overload protection. To guarantee simulation accuracy, we validate our implementation against a physical Linux testbed across 25 diverse Bandwidth-Delay Product (BDP) scenarios. Results demonstrate that our ns-3 model replicates real-world behavior, ensuring strict throughput fairness and queue delay isolation. Ultimately, this validated model equips the research community with a robust tool to evaluate and advance L4S performance across diverse network topologies, ranging from data centers to wireless home and office environments.

0.04

Building A CSFQ-Inspired Transport for Switched CXL Memory Pooling

cs.NI ·2026-08-22 ·Zerui Guo, Emily Shriver, Ming Liu
published similarity 0.62
Abstract

Emerging switched CXL memory pooling systems, albeit promising, suffer from significant performance interference due to the shared but performance-uncontrolled data path among concurrent memory streams between a host core and a remote DIMM. We systematically characterize a memory pooling appliance based on XConn's Apollo CXL switch and identify three issues: intra-host contention, in-fabric congestion, and unmanaged host-remote DIMM interaction. This paper presents a new transport layer, MemChannel, which provides the mchannel abstraction to manage end-to-end fabric bandwidth among competing memory flows and enable application-specific traffic for switched CXL memory pooling. Our key idea is to build a sender-driven, fabric-informed transport protocol, inspired by Core-Stateless Fair Queueing (CSFQ), that admits just the right amount of CXL requests to each mchannel based on the estimated core-to-CXL-DIMM bandwidth availability. To address CXL-induced idiosyncrasies, MemChannel introduces time-based rate control, host-side admission control, cross-host bookkeeping, new congestion signals, rate estimation based on the fluid model, and delay-based link-capacity adjustment. We build MemChannel from scratch and support unmodified applications. Evaluations over switched memory pooling demonstrate its effectiveness from performance-isolation, scalability, and multi-tenancy perspectives.

0.48

Towards Fair Resource Sharing in Agentic LLM Serving

cs.OS cs.AR ·2026-11-17 ·Sehyeon Lee, Yuchen Xia, Zhaoyang Hong, Jinrui Hu, Anuradha Pandey +3 more
similarity 0.61
Abstract · 15 pages, 14 figures, 2 tables

As LLM agents increasingly execute complex, multi-step tasks on shared inference infrastructure, resource contention across the end-to-end LLM serving stack causes severe and uneven slowdowns across agents. Existing agentic serving systems primarily optimize throughput and latency, with comparatively limited attention to fairness among concurrent agents. Although fairness has been studied in conventional LLM serving at the token and request levels, these approaches do not fully capture cumulative delays and stateful resource dependencies across successive LLM calls within an agent. Moreover, interference spans request routing, runtime scheduling, and hardware resources, making agent-level fairness challenging to define and optimize across an end-to-end serving stack. This paper presents a cross-layer framework to define agent-level fairness, attribute sources of unfairness, and optimize fairness while preserving throughput. To account for agents with widely varying execution demands, we build a theoretical foundation to define fairness using slowdown relative to standalone execution. Our empirical analysis reveals that routing, runtime scheduling, and hardware contention jointly determine fairness, with performance-oriented policies often leading to prolonged queuing delays. To optimize fairness, we introduce Duru, which monitors each agent's slowdown and uses this signal to guide admission and scheduling decisions across the router and runtime. Duru further alleviates starvation through admission exceptions and integrates slowdown-driven autoscaling to adhere to Service-Level Objectives (SLOs). Our evaluation across various benchmarks and hardware platforms shows that Duru reduces 99th-percentile normalized slowdown by a geometric mean of 44.4% (up to 79.5%) while delivering 91.8-127.4% of the state-of-the-art system's throughput.

0.09

Token Latency Fairness: Performance Isolation for Multi-Tenant LLM Serving

cs.DC cs.LG ·2026-09-16 ·Dev Bali, Soujanya Ponnapalli, Yichuan Wang, Natacha Crooks, Scott Shenker +1 more
similarity 0.60
Abstract

LLM serving is typically offered as a shared, multi-tenant service, where high-demand workloads from one client can cause latency SLO violations for others. Existing solutions for performance isolation equalize client throughput in the long run, for example through queueing and batching fairness. However, these approaches do not provide latency isolation guarantees; as a result, well-behaved clients can still experience significant degradation to their token-level latencies. In this paper, we present FairInference, which provides the novel δ-token fairness guarantee: for a well-behaved client, if a token is generated in d time units in isolation, it will be generated within d + δ time units in multi-tenant execution, providing strong latency isolation guarantees for LLM serving. To achieve this, FairInference addresses a key challenge of LLM serving: bounding delays from sharing GPU resources without support for fine-grained scheduling or resource allocation. In FairInference, the scheduler enforces per-token deadlines, while bounding the delays from GPU compute sharing and accounting for the additional delays introduced by the shared KV caching in GPU memory. We show that FairInference effectively bounds token-level latency spikes for well-behaved clients and improves overall throughput compared to state-of-the-art LLM serving systems.

0.46

PANEM: A Heuristic Latency Model

cs.AR cs.PF ·2026-08-28 ·Lucas Crowthers, Mahesh Madhav
published similarity 0.60
Abstract · 12 pages, 23 figures, Presented at the 2026 IEEE International Symposium on Workload Characterization (IISWC 2026), Boulder, Colorado

Accurate pre-silicon memory modeling is essential for achieving meaningful representation of workloads on cloud-class many-core processors. Existing options force a poor tradeoff between fidelity and speed: fixed-latency models are fast but misleading, while cycle-accurate DRAM models are costly and difficult to scale across large study spaces or onto single-core environments. This paper presents PANEM, a lightweight event-driven heuristic model that has guided four generations of commercial server-core development at Ampere Computing. PANEM converts bandwidth-latency characterization data into a dynamic request-bytes/latency response, allowing miss latency to adapt to transient demand, queuing pressure, and read/write mix during simulation. Integrated into a single-core flow with configurable system-loading assumptions, PANEM enables realistic bandwidth constraints and contention-aware latency behavior without sacrificing throughput. Across a broad cloud workload trace suite, PANEM avoids the optimistic and pessimistic biases of fixed-latency baselines, yields more reliable conclusions for prefetching and dynamic throttling studies, and materially improves core-resource sizing decisions. These results show that a calibrated, contention-aware abstraction can deliver practical predictive value for industrial design-space exploration at simulation costs similar to fixed-latency models.

0.46

Beyond the Limits: Flexible and Congestion-Aware Cluster Scheduling for the Cloud

cs.DC ·2026-08-10 ·Oliver Larsson, Thijs Metsch, Cristian Klein, Erik Elmroth
similarity 0.60
Abstract · 10 pages, 9 figures, 3 tables

Workload scheduling in cloud environments often relies on simplistic assumptions about application resource needs and hardware utilization. Overlooking application-level performance objectives and hardware resource contention that leads to inefficient resource usage and degraded performance. This paper addresses two key limitations of current approaches. First, unnecessarily strict enforcement of service level objectives (SLOs) often leads to resource underutilization and poor energy efficiency. Second, lack of congestion awareness in shared resources such as last-level cache (LLC) and memory bandwidth. In this paper, we propose two complementary strategies to address these limitations: (i) integrating soft SLO limits that allow controlled overcommitment and tolerate minor, transient violations to improve cluster efficiency, and (ii) introducing resource-aware scheduling and rescheduling based on real-time congestion insights for shared resources such as last-level cache (LLC) and memory bandwidth. Our results show that soft SLO limits reduce corrective rescheduling actions by 49% compared to hard-limit approaches while maintaining acceptable performance guarantees. Additionally, resource-aware scheduling decreases node-level congestion by 8% and further mitigates SLO violations, demonstrating the effectiveness of incorporating application-level flexibility and hardware-level insights into scheduling and rescheduling decisions.

0.42

Completion-Path Credits: Multi-Resource Control for Scale-Up Fabrics

cs.NI ·2026-08-18 ·Fan Yang, Jiaqi Liu, Tao Jiang, Zhan Wang
similarity 0.59
Abstract · 10 pages, 5 figures

Scale-up fabrics connecting GPUs and AI accelerators carry tensor transfers together with remote reads, writes, atomics, and notifications over shared target-side receiver resources. Byte-denominated credits protect link buffers and streaming HBM traffic, but poorly represent small operations dominated by Atomic execution or response injection. This paper presents SemaCredit, a receiver controller that admits each remote-memory operation against a vector of target-resource demands and returns each component when its corresponding HBM, Atomic, or response stage completes. In a deterministic event simulator with multipath queues, eight HBM partitions, a serialized Atomic engine, and a response engine, SemaCredit matches a strong per-resource byte baseline on HBM-hotspot traffic while reducing small-operation P99 latency by 52.4% under Atomic contention and 10.2% under response incast. Application-shaped mixes show 57.7% and 14.5% P99 latency improvements for AllReduce-shaped and remote-read-shaped traffic while matching byte credits on HBM-dominated MoE traffic.

0.12

PRICE: Pricing-based Resource Incentives for Quality-of-Result-aware Computing at the Edge

cs.DC ·2026-08-21 ·Uwe Gropengießer, Sebastian Frenz, Max Mühlhäuser
published similarity 0.59
Abstract

Edge nodes are capacity-constrained by design, yet many edge workloads can trade result quality for resource efficiency at runtime. Existing edge pricing mechanisms largely treat requests as fixed-configuration submissions and rarely exploit per-request quality flexibility under overload. We present PRICE, an incentive mechanism that couples a utilization-dependent price signal to per-request quality selection. As utilization increases, rising acceptance prices make resource-intensive variants less likely to be selected, shifting accepted requests toward lighter execution and allowing the node to serve significantly more requests while operating near capacity. Evaluation on real hardware under sustained overload shows that PRICE outperforms both fixed-allocation and dynamic-pricing baselines in accepted throughput and CPU utilization. The results are robust across pricing function families, task-duration distributions, and client populations. Result-quality flexibility is a powerful but underused control dimension for overload management at the edge, and pricing is an effective mechanism to exploit it.

0.24