CS Paper Radar

LatencyLab: A DPDK-Based P4 Pipeline Latency Measurement Framework for FPGA SmartNICs

Pavani Kuppili, Zhaoyang Han, Yicheng Qian, Suranga Handagala, Michael Zink, Miriam Leeser, Robert Ricci

cs.DC ·submitted 2026-09-30 15:38 UTC · v1
published

Abstract

P4-programmable FPGA SmartNICs place packet processing directly on the wire, but open FPGA P4 toolflows do not expose timestamping at the pipeline boundary, so the latency a P4 program adds on the target FPGA is rarely measured. This paper presents LatencyLab, a DPDK-based measurement framework for FPGA P4 pipeline latency that needs neither PHC/PTP support on the datapath nor clock synchronization. The FPGA's two ports share a network segment, so the switch multicasts a copy of each probe packet to both: one copy passes through the VitisNetP4 pipeline, the other through a matched bypass path. A kernel-bypass DPDK receiver busy-polls both ports and timestamps every packet with the CPU timestamp counter (TSC) as it is retrieved from the NIC's receive circular buffer. The arrival-time difference of the two copies isolates the pipeline latency after calibration against a null bitstream carrying the same traffic; transmit time cancel in the subtraction. We evaluate four VitisNetP4 programs on an AMD Alveo U280, probing each with a 20,000-packet trace measured ten times per session over five independent sessions, all TSC-timestamped and reflected for hardware timestamping. The measured latency distributions are tight and reproducible: 99% of packets fall within 20 ns of the median, session medians repeating within 1 to 2 ns (FiveTuple 107/137 ns, Forward 149 ns, RemoveHeader 177 ns, Checksum 364 ns at 250 MHz). Two independent checks agree with the framework: a kernel-free reflector returns every probe pair to a ConnectX-5 NIC whose adapter clock reproduces the measured distributions within a few nanoseconds, quantile by quantile, and every measured packet falls 18 to 22 clock cycles below the vendor's worst-case latency bound.

Comment
This paper appeared in the 30th Annual IEEE High Performance Extreme Computing (HPEC) https://ieee-hpec.org/ where it won an outstanding paper award

Closest papers

Nearest neighbours by meaning across the whole index.

Gateway-Free Kafka Streaming from FPGA-Based Detectors: A Bare-Metal Pipelined Producer on Zynq-7000

cs.NI cs.AR ·2026-11-17 ·Fernanda Zapata Bascuñán, Matías Hampel
published similarity 0.57
Abstract · Accepted for pucblishing at IEEE Explore proceedings of ARGENCON 2026

We present a bare-metal Apache Kafka producer implemented on a Zynq-7000 System-on-Chip (SoC) for event-driven data acquisition. Events are generated in Programmable Logic (PL), transferred to the Processing System (PS) through AXI4-Stream and AXI DMA, and published to a standard Kafka broker without an intermediate gateway computer. The PS constructs ProduceRequest messages and transmits them through the lwIP raw TCP/IP API. A configurable pipeline limits the number of outstanding Kafka requests. Pipeline depths of D = 1, 2, 4, and 8 were evaluated with two independent DMA sources. For each depth, 20 independent runs of 5,000 events were acquired, for 400,000 event records in total. Each event carries a 32-bit frame counter and a 32-bit PL timestamp. PS-side instrumentation records DMA start and completion, publication, successful local tcp_write() acceptance, and reception of the corresponding Kafka acknowledgement. Measurement records are exported to a separate Kafka topic after the timed event stream has completed. Each of the 80 runs contained 5,000 records with frame values from 0 to 4,999, with no sequence discontinuities in the exported metric records and no recorded Kafka errors. Mean operational throughput was approximately 5.3 kevents/s at D = 1, 9.4-9.6 kevents/s at D = 2, 16.2 kevents/s at D = 4, and 16.8-17.5 kevents/s at D = 8. The median publish-to-acknowledgement interval was 1.62-1.64 ms at D = 1, 0.98 ms at D = 2, 0.66 ms at D = 4, and 0.83-0.86 ms at D = 8. At D = 4, the corresponding P95 values were 0.81-0.83 ms.

0.35

High Speed Network Processing for Scientific Computing in the Open Cloud Testbed

cs.NI cs.DC ·2026-10-05 ·Miriam Leeser, Michael Zink, Suranga Handagala, Carlos Ruben Dell'Aquila, Verena Martinez Outschoorn +1 more
published similarity 0.55
Abstract · Paper accepted to the Twelfth International Workshop on Heterogeneous High-performance Reconfigurable Computing (H2RC'26) Held in conjunctio…

This paper presents an FPGA-based SmartNIC architecture designed to address critical network challenges in data-intensive scientific computing. As scientific workflows increasingly demand high-bandwidth data movement and real-time monitoring across wide-area networks, the Open Cloud Testbed (OCT) provides an ideal platform for investigating in-network processing solutions at line rate. Our approach focuses on implementing network visibility, optimization, and orchestration capabilities through FPGA acceleration. The proposed system leverages P4-programmable data-plane functions to support packet parsing, marking, telemetry extraction, traffic steering, and machine learning inference, all without requiring host CPU intervention or modifications to application servers. As a concrete example, our framework integrates the SciTags standard, which enables associating network traffic with scientific context through packet marking. We implement a high-performance network bridge that counts packets by their SciTag values, as identified in the IPv6 Flow Label. Experimental evaluation on AMD U280 FPGAs in OCT demonstrates the feasibility of this approach, showing imperceptible processing latency and line-rate packet handling, while accurately monitoring scientific data flows.

0.22

An Experimental Evaluation of Scheduled Transmission and Hardware Timestamping on the Intel E830 NIC

cs.NI ·2026-10-08 ·Takahiro Hirofuchi
similarity 0.55
Abstract

High-precision packet transmission is increasingly important in deterministic networking applications, including 5G fronthaul and Time-Sensitive Networking (TSN). Some recent high-speed network interface cards (NICs), including the Intel E830 and NVIDIA ConnectX series, support hardware-assisted transmission at specified times and hardware timestamping of received and transmitted frames. Despite the relevance of these capabilities to the stringent timing requirements of 5G fronthaul, little public information is available on their timing accuracy and performance characteristics. As a companion to our previous study of scheduled transmission on the NVIDIA ConnectX series, this paper experimentally characterizes the scheduled transmission and hardware timestamping capabilities of the Intel E830 NIC. Using an FPGA-based measurement platform with deterministic frame generation and nanosecond-resolution timestamping, we evaluate receive and transmit hardware timestamping and scheduled-transmission accuracy. The results show only several-nanosecond timing variation in hardware timestamping. Receive timestamp-derived frame intervals showed a peak-to-peak variation of up to 6 ns, while the transmit timestamp comparison showed a residual range of 4.6 ns after clock-drift compensation. Approximately 99% of the observed frame intervals were within $\pm$300 ns of the target interval, with occasional outliers of up to approximately 2 us. These results indicate that scheduled transmission can support applications with timing requirements on the order of several tens of microseconds, such as 5G fronthaul, but may be insufficient for TSN applications requiring transmission timing accuracy on the order of a few to a few tens of nanoseconds.

0.26

Decentralized Latency Models and Information Flow for TSN Shapers - A Tutorial

cs.NI ·2026-10-08 ·Alexej Grigorjew, Stefan Geissler, Jürgen Schmitt, Tobias Hossfeld
similarity 0.54
Abstract · This work has been submitted to IEEE COMST for possible publication. Copyright may be transferred without notice, after which this version m…

Recent advancements in the fields of industrial automation and in-vehicle communication have also increased the requirements on their underlying networks. Traditional field bus technologies are being replaced by regular Ethernet devices with features from the IEEE Time-Sensitive Networking working group. Their standards are designed to achieve zero congestion loss and deterministic latency bounds, but the specific guarantees can differ depending on the shapers used and how the bounds are calculated. After several contributions to the standardization efforts, this work presents a number of decentralized latency models that can be applied locally on the switches, without requiring global knowledge from the rest of the network. It includes a formal definition of delay segments that allows to incorporate interoperability between different shapers and latency models in the future. It presents pseudo-code suggestions for the latency bounds computation of Strict Priority, Credit-Based Shaping, Asynchronous Traffic Shaping, and Cyclic Queuing and Forwarding, all of which can be applied locally on the devices. Finally, five models were implemented and evaluated with respect to their efficiency in terms of latency, jitter, and number of accepted streams. The results illustrate how different shapers are suited for different networks and traffic distributions.

0.32

Packet iSlip

cs.NI ·2026-09-25 ·Marc Mosko
similarity 0.53
Abstract

This paper examines input/output buffered crossbar switches under combined packet and cell data. Our switch architecture uses input buffering with Virtual Output Queues to avoid Head of Line Blocking. The switch fabric is a crossbar with no speedup. We use a modified iSlip [McKeown] crossbar scheduler geared towards packet data, called piSlip. Cell ports are largely unmodified from standard iSlip behavior. For packet output ports, we introduce changes to the grant pointer which minimizes output latency caused by packet reassembly. Our model uses several output states, including packet cut-through. From simulation results, we show that piSlip with virtual cut-through offers latency characteristics significantly better than unmodified iSlip with similar packet port interfaces. Simulation further shows that piSlip and iSlip have similar maximum and average buffering requirements.

0.26

Latency-Neutral Passive Cyber Monitoring for Grid-Edge Operational Technology Networks

eess.SY cs.CR ·2026-10-20 ·Damian Byrne, Emad Abukhousa, Jason Reed, Saman Zonouz
published similarity 0.53
Abstract · 5 pages, 4 figures, 2 tables. Accepted for presentation at the IEEE PES Grid Edge Technologies Conference and Exposition 2027

Deterministic industrial networks often present an operational trade-off between maintaining microsecond timing of primary control loops and capturing security telemetry. This work investigates the engineering feasibility of the Operational Technology Supervisor (OTSV), a three-tier passive monitoring framework designed to resolve this visibility challenge within timing-sensitive grid-edge infrastructure. By utilizing a Layer-3-unconfigured passive ingestion mode on bare-metal hardware via a physical switched port analyzer (SPAN) loop, the deep packet inspection engine is isolated from the production network, implementing a software-enforced unidirectional export policy. Performance validation conducted over a continuous audit comprising 32,927 active-monitoring round-trip time (RTT) samples yielded stable latency distributions across a laboratory power-monitoring loop running Modbus TCP and IEC 61850 MMS protocols. The measured latency delta between the unmonitored baseline and the active monitoring state was less than 0.12 milliseconds, supporting the feasibility of continuous high-fidelity security observation alongside stringent grid-edge metering timing bounds without measurably degrading the monitored reporting path.

0.31

Hardwired Distributed Processing Units at the Sensor: Truly Parallel Vision Pre-Processing that Frees the Host CPU

cs.DC cs.AReess.IV ·2026-10-28 ·Raj Mali
similarity 0.52
Abstract · 7 pages, 6 figures, 7 tables. Companion to arXiv:2610.33107

Conventional vision pipelines route every pixel to a host processor, which must then wait for a complete frame before it can act and devote part of each frame period to routine pre-processing. We examine an alternative organisation in which this work is distributed across hardwired distributed processing units (HDPUs): small, fixed datapaths attached directly to the sensor, each committed to a single always-on task, operating truly in parallel on the same pixel stream and delivering compact results to the host. From one library of 3x3 operators sharing a common two-clock latency, we construct three HDPUs (Canny edges, Harris corners, and a line tracker that issues a steering decision) and run them side by side on a single camera stream in a low-cost Artix-7 FPGA, at a cost of 2,568 LUTs and an estimated 0.318 W. Every stage of every unit is bit-exact with its software model. In a full-system simulation with a datasheet camera model, each decision leaves its HDPU within 11.03 camera lines of the pixel that produced it, 22-67x sooner than the earliest moment at which a frame-based program on the host could respond; the HDPU latency varies by no more than 17 ns, whereas the host's own run times reached 12.5x their median. Offloading the three tasks returns 18-30% of a core to the host in every frame period at 640x480, and 63% of the frame period, with all threads in use, at 1080p. The cost of the units grows with frame width only through their line buffers, and in a closed-loop model of a line-following vehicle the HDPU raises the highest speed at which the line is held from 4.75 to 10.00 m/s.

0.42

From Isolated Protocols to Integrated Pipelines: Benchmarking IoT Communication in the MASA Living Lab

cs.NI ·2026-11-11 ·Antonio Solida, Enrico Rossini, Marcello Pietri, Marco Picone, Carlo Augusto Grazia
publishedbenchmark similarity 0.52
Abstract · Accepted for publication at the IEEE 2027 Consumer Communications and Networking Conference; track T3: Testbeds, Experimentation and Dataset…

The performance of real-time IoT communication systems depends not only on individual transport protocols, serialization formats, and data-processing stages, but also on how these components interact within the complete communication pipeline. Selecting an appropriate communication stack is therefore application-dependent and requires considering processing capabilities, network characteristics, communication patterns, and reliability requirements. This paper provides an experimental, evidence-based analysis of these interactions through a controlled benchmark of integrated communication pipelines deployed in the MASA living lab. We jointly evaluate local processing overhead, end-to-end latency, jitter, throughput, and delivery reliability across different combinations of transport, messaging, serialization, and compression mechanisms. Results show that component-level processing costs do not directly predict end-to-end performance. In particular, differences of only tens of microseconds in serialization or compression costs can coexist with several-millisecond differences in end-to-end latency, while middleware buffering and transport behavior substantially affect timing performance. The results demonstrate that communication-stack components should be evaluated as integrated pipelines and under representative operating loads rather than ranked independently.

0.32