CS Paper Radar

The Cost of Saying Hello: An Empirical Study of Handshake Overhead in AI Agent Communication Protocols

Pranav Varshney

cs.AI cs.LG ·submitted 2026-08-12 03:51 UTC · v1
benchmarkcode/data

Abstract

Every interaction between AI agents begins with a handshake: the initiating agent must locate its counterpart, learn what it can do, and establish a session before the first useful message can be sent. Surveys of agent communication protocols name handshake overhead as an efficiency criterion, yet none report measurements. This paper presents the first empirical, phase-decomposed benchmark of the handshake in three major agent protocols: MCP (in both its 2025 initialize and 2026 discover eras), A2A, and ACP. Using each protocol's official SDK, we instrument every request for round trips, per-phase latency, wire bytes, and token cost across cold-start, warm-repeat, capability-mismatch, and degraded-metadata scenarios (N=50 per cell, on loopback and under an emulated 50 ms RTT). Round-trip count dominates handshake cost. A legacy MCP handshake takes three round trips and 230 ms at 50 ms RTT; A2A needs one round trip and 61.0 ms. Round trips are not the whole bill: when an LLM reads the counterpart's metadata, handshake cost scales with the counterpart's capability inventory at 24-107 tokens per capability, so a 200-capability counterpart spends 11% of a 200k-token context window before the agent reads its own task. MCP's own 2026 revision cuts its handshake from three round trips to two and, with era pinning, to zero on warm reconnects, protocol evolution that corroborates the cost we measure. We release the full harness, raw data, and analysis at https://github.com/pvarshh/handshake-benchmark.

Comment
15 pages, 4 figures, 4 tables. Harness, raw per-run data, and analysis: https://github.com/pvarshh/handshake-benchmark

Closest papers

Nearest neighbours by meaning across the whole index.

The Cost of a Hop: Benchmarking NLIP and A2A

cs.AI ·2026-10-02 ·Ranjan Sinha, Anindita Das, Ashika Anand Babu, Hari Palleti
similarity 0.72
Abstract

Autonomous agents built on Large Language Models (LLMs) need standardized protocols to interoperate across systems. Several now exist (A2A, MCP, ACP, ANP, NLIP), but the Natural Language Interaction Protocol (NLIP) has not appeared in any controlled performance study, and no work has measured where an agent protocol's latency is spent. We compare NLIP and the Agent-to-Agent (A2A) protocol empirically, decomposing latency into message creation, connection, and send phases across three independent hardware environments. For lightweight coordination, NLIP is 8.4-9.6x faster than the baseline A2A SDK implementation on two environments and about 4x on a third; the direction of the advantage is consistent, its magnitude depends on the hardware. The advantage is stage-specific: for the end-to-end pipeline, where LLM inference dominates, the protocols are near parity. The difference comes almost entirely from connection setup. To test A2A at its best, we also ran A2A SDK with connection caching enabled; caching narrows its gap with NLIP by a hardware-dependent amount, from 2.75x on one machine to near-parity on faster hardware, where at scale a cache-optimized A2A-SDK matches NLIP. We report these as measured conditions without a single causal account of the residual send-phase cost. Against the more optimized Python-A2A, NLIP leads by about 4x on the same stage. We close with a protocol-selection guide keyed to workload characteristics.

0.43

RouterWorld: Scaling Conversational AI with Latency-Aware LLM Routing

cs.AI cs.CLcs.LG ·2026-10-19 ·Jinyang Li, Denis Emelin, Simone Tedeschi, Weiwei Cheng, Cristina Morariu +4 more
benchmarkcode/data similarity 0.60
Abstract · 28 pages, pre-print

Conversational AI systems built on Large Language Models (LLMs) require routing mechanisms that balance answer quality and response latency at scale. However, existing LLM routing benchmarks assume full observability of all candidate model outcomes, under-emphasize latency as a first-class metric, and fragment evaluation across heterogeneous protocols misaligned with real-world settings. We introduce RouterWorld, a comprehensive benchmark for LLM routing in conversational GenAI. RouterWorld unifies 371k single-turn responses and 76k multi-turn agentic trajectories, adopts natural user interactions with a standardized LLM-as-a-judge protocol, and decouples efficiency from transient pricing via hardware-grounded latency annotations. Crucially, RouterWorld formalizes routing under partial observability through two setups: Cold-Start, which restricts learning to sparse, static logs, and Voyager, which models continuous data streams where routing decisions shape future training distributions. Evaluating 13 routing baselines, we find that most routers over-emphasize accuracy while ignoring latency under partial observability. Code and benchmark data are publicly available.

0.29

Can Agent Harnesses and Inference Engines Hear Each Other? The HEAR Protocol for Agentic LLM Serving

cs.AI ·2026-10-05 ·Jiaqi Zhao, Haodong Chen, Jitai Hao, Wei Zhao, Jinghao Pang +2 more
similarity 0.60
Abstract · Jiaqi Zhao, Haodong Chen, and Jitai Hao contributed equally

LLM agents increasingly execute complex workflows involving multi-turn reasoning, tool use, and parallel agents. Efficient serving requires decisions that span two layers with complementary information: the agent harness understands workflow dependencies, context lifecycles, and execution objectives, whereas the inference engine observes request queues, KV-cache state, resource pressure, and execution capabilities. Existing interfaces do not systematically connect these views, limiting workflow-aware execution. HEAR, a bidirectional Harness--Engine Pairing protocol for agentic LLM serving. HEAR standardizes how the harness communicates workflow intent and execution requirements and how the engine returns runtime state, capabilities, and outcomes. By separating protocol semantics from optimization policies, HEAR supports diverse coordination strategies without changing workflow or model semantics. We instantiate HEAR for online cache-aware runtime coordination and workload-aware execution-mode selection for agent roles. Across four conversational and research-agent benchmarks under memory-constrained, concurrent serving, HEAR achieves a $1.61\times$ batch speedup and reduces median time-to-first-token by $2.23\times$ on SCBench. Mooncake shows that workflow intent and live engine state provide complementary benefits across load regimes. On BrowseComp-Plus and DeepResearchBench, workload-specific configurations yield $1.23\times$ and $2.45\times$ end-to-end speedups, respectively, without observed task-quality degradation. These results establish HEAR as a reusable coordination substrate for efficient agentic LLM serving.

0.14

Characterizing AI Service Traffic for 6G: An Open Testbed and Initial Results from 3GPP SA4

cs.NI ·2026-11-04 ·Imed Bouazizi
similarity 0.59
Abstract · 19 pages

The design of sixth generation (6G) mobile systems requires measurements of artificial intelligence (AI) service traffic under controlled network conditions. The 3rd Generation Partnership Project (3GPP) Service and System Aspects Working Group 4 (SA4) documents such measurements in draft Technical Report 26.870. In this paper, we describe the open testbed used for the study and analyze the results recorded in version 0.6.2 and its companion workbook. The testbed combines network emulation, packet capture, application measurements, and scenario execution against hosted and self-hosted models. The reported uplink to downlink volume ratios span more than six orders of magnitude. A personal assistant agent has a ratio of 0.007 at its local control interface and 0.50 on its wide-area egress. Deterministic agent signaling follows the configured round-trip delay. In a tokenized video experiment with 125 questions per profile, the number of correct answers changes by at most three through 10% independent uplink loss. Larger losses reduce accuracy, but the available repetitions do not establish a tolerance threshold. We also analyze emulator behavior that affects the interpretation of the study. Per-packet jitter introduces reordering, correlated random-loss settings suppress rare loss events, and a Gilbert-Elliott parameter mapping changes the stationary loss rate. The results support characterization by scenario, measurement layer, and versioned network configuration.

0.24

Real-Time dApps for AI-RAN: Measured Interface Requirements for Inline PHY and Slot-Level Control

cs.NI cs.DCeess.SP ·2026-09-07 ·Timothy O'Shea, Matthew Pennybacker, Andriy Kharchenko
code/data similarity 0.58
Abstract · 13 pages, 7 figures, 6 tables. Preview version accompanying a preview release of the OCUDU dApp platform; suite, corpus, traces, and sources…

Distributed applications (dApps) bring AI to the microsecond-to-millisecond band beside the 5G distributed unit (DU), but every public dApp framework realizes them the same way: an external process that receives an indication and returns a control message. That boundary is right for sensing and advisory workloads. It cannot express a neural receiver that must finish inside a slot, and it cannot hold a scheduling decision the MAC is waiting on. This paper asks what a dApp interface must deliver, in latency and bandwidth, for the AI-RAN use cases now filed under that name to be realized as dApps at all. It treats the coupling between a dApp and the DU as a design axis with two legitimate ends, a C ABI inside the DU process and a protocol-standard E3AP association over SCTP, and places the three OCUDU dApp classes on it. An audited corpus of 39 runtime AI-RAN use cases, sized by 5G NR timing, shows that more than half cannot cross the observer boundary: inline PHY work because an indication has no return path into the same slot, and bounded control because of the tail under load. Measuring the mechanisms each framework actually uses, on a quiet host and with a live cell on the air, shows that every carrier meets a 100 us control deadline on an idle host and that only the in-process paths still do once the DU is running. The asynchronous use cases remain feasible at every position, so the protocol-standard end is kept as a first-class option; what in-process placement adds for them is accountable staleness and an observation-to-decision path with no message on it. These measurements derive the released ABI, a stream for inline work, a validated call for bounded control, and a choice of lease, supervised ring, or portable E3 client for observation, and four dApps of all three classes are validated together on one over-the-air cell.

0.59

AgentPerfBench: A Benchmarking and Evaluation Suite for Inference Performance of Agentic LLMs

cs.LG ·2026-09-28 ·Cheuk Hang Lau, Zeyu Cao, Kevin Wong Cheuk Yin, Yao Lai, Haoran Wu +4 more
benchmark similarity 0.57
Abstract

The optimization of LLM serving engines, such as vLLM and SGLang, is largely benchmark-driven: optimizations, scheduling policies, hardware and system designs are all selected based on representative workloads. However, a significant mismatch has emerged in the agentic era. Existing benchmarks primarily focus on simple single-turn chatbot workloads. LLM applications are increasingly agentic: coding agents, terminal execution systems, and tool-use agents issue multi-turn requests with growing context lengths. We introduce AgentPerfBench, a benchmark suite for agentic inference. It uses real traces from agentic benchmarks, such as SWE-Bench and TerminalBench, alongside standard chat baselines. This enables benchmarking of models on multi-turn tasks involving tool calling, skill utilization, and increasing context lengths. AgentPerfBench also samples from empirical distributions of input length, output length, and turn count derived from the real traces, generating representative synthetic profiles for cheap and accurate measurements on new hardware. In addition, we further find that several existing benchmarks fail to accurately reflect real hardware performance for two key reasons: 1) they do not account for realistic context-length growth, and 2) they measure inference performance without operating at hardware saturation. We discuss these issues in detail and provide rich kernel-level Nsight Compute (NCU) traces to construct a new multi-dimensional roofline model that captures hardware-system limitations in both memory bandwidth and memory capacity footprint. The benchmarking suite then includes automated scripts to identify potential bottleneck conditions on emerging hardware when evaluated with diverse agentic traces. Together, these contributions quantify the chat-to-agentic gap in current inference benchmarks and characterise per-kernel GPU resource utilisation via roofline analysis.

0.36

Joi-Bench-Task: Benchmarking Task Coordination in Full-duplex Voice Agents through Interactive Simulation

cs.AI ·2026-11-09 ·Xiangheng He, Mingjun Zhao, Zhehao Fan, Zhengjia Li, Xiaopeng Sun +3 more
benchmark similarity 0.57
Abstract

Full-duplex voice agents enable natural, hands-free interaction, allowing users to refine requests, redirect ongoing work, and seek clarification while the agent is speaking. However, existing voice-agent benchmarks insufficiently isolate coordination between task execution and conversation. We introduce a benchmark that assesses this coordination by examining whether agents track what is being executed, what has been completed, what has been communicated, and what remains to be reported as the conversation evolves. We propose an interactive user simulator that introduces configurable interventions at designated timing in tool execution and spoken result delivery, supporting controllable and reproducible evaluation conditions. We evaluate five leading commercial voice agents along three dimensions: execution responsiveness, coordination under request changes, and coordination under interruptions. Agent evaluations reveal persistent coordination failures. Under request changes, the highest result completion rate is only 55%, indicating difficulty for current voice agents adapting to new requests while preserving earlier obligations. After user interruption, four of the five agents exhibit average information omission rates of at least 23%. These findings highlight task coordination as a central challenge for current voice agents in complex, evolving conversations.

0.16

ToolSched: Resource-Efficient Tool Scheduling for Shared-Host LLM Agents

cs.OS ·2026-11-03 ·Shucheng Zou, Yuhao Gu, Long Chen, Zhongchun Zheng, BinZe Xie +1 more
similarity 0.57
Abstract

Local LLM agents often work on shared hosts. Unlike multi-tenant cloud clusters, these hosts often have tightly bounded memory, CPU, and I/O resources. Concurrent agents frequently suffer from resource contention, leading to failures such as OOM and reduced resource efficiency. Considering that agents' resource demands are bursty and shell command delay is imperceptible to agents, there is a huge scheduling opportunity to arrange when and in what order commands are actually executed. Thus, we present ToolSched, a Shell MCP Server for shared local hosts that serves multiple agents and schedules their operations. ToolSched embeds a resource model into the MCP interface: agents estimate resource demands along with command requests. These hints, together with real-time host resource conditions, determine when commands are actually executed. Internally, ToolSched applies a scheduling algorithm to stagger commands, reducing contention and improving resource utilization. Prediction correction and checkpoint-based recovery mitigate estimation bias and resource errors. Our evaluation on Terminal Bench shows that, compared with unscheduled execution, ToolSched improves resource utilization by 148.4% and end-to-end command throughput by 29.4%, demonstrating its effectiveness and practicality.

0.20