CS Paper Radar

Perception-Aware Communication Middleware for Distributed Visual Perception in UAV Swarms

Manveen Kaur, Kevin Loi, Ifunanya Okafor, Daniel Ng, Joseph Lucey-Renteria

cs.MA ·submitted 2026-09-21 17:51 UTC · v1

Abstract

Unmanned Aerial Vehicle (UAV) swarms increasingly support safety-critical applications that rely on distributed visual perception. Meeting the low-latency requirements of these applications can require perception models to execute within the swarm on inference-capable UAVs, creating a need for efficient UAV-to-UAV transport of high-bandwidth perception data. However, the Quality-of-Service (QoS) requirements of perception differ from conventional packet-level QoS; successful delivery of individual packets does not ensure that a complete, timely, and usable image is available for inference. We present a novel perception-aware communication middleware that treats complete perception-data samples as the communication objects for which QoS must be satisfied. The middleware extends a lightweight UDP broker-based publish-subscribe architecture with perception-specific services, including image fragmentation and reconstruction, concurrent packet transmission, priority-aware scheduling, and image quality assessment. The middleware is evaluated on a heterogeneous hardware testbed emulating a UAV swarm using YOLOv8n object detection. Experimental results demonstrate low end-to-end application latency, substantially higher throughput than a lightweight UDP broker, effective prioritization of perception traffic under increasing background load, and mitigation of object-detection degradation through middleware-level image quality assessment. This work provides an initial framework for integrating AI-specific data handling into communication middleware to support emerging distributed AI applications in multi-agent mobile cyber-physical systems.

Closest papers

Nearest neighbours by meaning across the whole index.

Cloud, Edge, or Split? Profiling Onboard and Split Vision-Language Model Deployment for Drone AI

cs.DC ·2026-09-21 ·Zoha Azimi, Reza Farahani, Schahram Dustdar, Christian Timmerer
published similarity 0.61
Abstract · 8 pages, 10 figures, and 3 tables. Accepted as a full paper in iEdge 2026

Vision-Language Models (VLMs) enable edge devices like unmanned aerial vehicles (UAVs) to interpret visual observations and reason about complex environments using natural-language instructions. However, their practical deployment remains challenging as onboard inference is constrained by limited computational, memory, and energy resources, whereas cloud-based inference introduces communication latency, bandwidth overhead, and dependence on network connectivity. To address these limitations, split computing offers a promising alternative by partitioning VLM inference between the resource-constrained UAVs and more capable remote servers. However, the performance trade-offs among fully onboard, cloud-based, and split-computing architectures for lightweight VLMs have not yet been systematically profiled. This paper benchmarks these three deployment paradigms using SmolVLM-256M as a representative lightweight VLM. We quantify their inference latency, computational resource utilization, communication overhead, and energy consumption across varying image resolutions and network conditions. Our results show that no deployment strategy is universally optimal; instead, the preferred strategy depends on the interaction between network conditions and input image resolution.

0.52

Designing, Deployment and Field Testing of C2Stack for Networked Intelligent Software-Defined UAVs

eess.SY ·2026-08-28 ·Maxwell McManus, Zhaoxi Zhang, Sidharth Santhi Nivas, Yuqing Cui, Prem Sagar Pattanshetty Vasanth Kumar +6 more
top venue similarity 0.59
Abstract

Unmanned Aerial Vehicles (UAVs) are emerging as critical enablers of next-generation wireless networking and autonomous systems. Despite their potential, deploying and testing networked UAV systems in real-world environments remains challenging, largely due to the absence of well-developed, end-to-end, ready-to-use protocol stacks. To fill this gap, we present C2Stack, a configurable protocol stack and experimental framework designed for real-time control, evaluation, and optimization of UAV networks. C2Stack incorporates a modular control plane, referred to as the~C2Stack Network Operating System (CNOS), alongside a programmable data plane that exposes APIs for cross-layer algorithm development, digital twin integration, and autonomous swarm control. In this article, we share our experience with the deployment and testing of C2Stack. We implemented C2Stack on a custom UAV swarm platform that integrates multiprocessor system-on-chip (MPSoC) radios with Intel NUC computing modules, enabling interoperability with various RF front ends. Field trials were conducted in both netted environments and large-scale outdoor test ranges, focusing on two representative use cases: (i) network utility maximization through online reinforcement learning, and (ii) collaborative interference source localization. The experiments demonstrate the feasibility of real-time, data-driven optimization in dynamic aerial environments, while also revealing practical challenges in field deployments of networked UAV systems, including power constraints, sensing limitations, and deployment logistics. We have made C2Stack source code available to the community under the MIT License, with the goal of establishing it as a foundational framework for experimental research on intelligent networked aerial systems.

0.37

Fleets Need a Context Plane: Rethinking Cooperative Perception for Autonomous Drones

cs.RO cs.DCcs.ET ·2026-09-01 ·Liangkai Liu, Xiaoxiao Wu
similarity 0.58
Abstract · 7 pages, 3 figures

Cooperative perception allows a drone fleet to combine observations from multiple viewpoints. However, existing systems typically fix their feature-sharing policies at design time or adapt to only one context signal. This is a poor fit for aerial fleets, whose missions, bandwidth, formation geometry, and scene coverage can change during flight. We quantify the cost of context-blind sharing on UAV3D by controlling feature exchange at evaluation time using a released DiscoNet checkpoint, without retraining. Mission-aware sharing matches full-sharing accuracy while using only 5-10% of the bytes. The best tested peer selection policy changes with the byte budget, and choosing the wrong policy loses up to 7.7 AP. Moreover, under a constrained budget, two policies with the same full-scene accuracy differ by 5.9 AP within the mission region, showing that multiple context axes must be considered jointly. We therefore propose the context plane, a bounded, structured interface for runtime context. Each drone publishes a descriptor of at most 1 KB at 10 Hz, and lightweight, replaceable policies use the fleet context to decide what each drone computes, shares, and fuses. Existing sharing schemes become fixed policies within this interface. In our ROS 2 prototype on a Jetson AGX Orin, the context plane uses approximately 0.01% of the data-plane bandwidth, and each policy decision takes 0.10 ms. These results show that an explicit context interface can support low-overhead runtime adaptation without modifying or retraining the perception model.

0.35

Adaptive Semantic Communication with Residual Quantization for Resilient Vehicular Networks

cs.NI ·2026-10-07 ·Xuanhao Luo, Ruichen Gao, Sandeep Sreenivasan, Zhizhen Li, Yuchen Liu
similarity 0.57
Abstract

Vehicular networks require timely visual information exchange to support safety-critical applications such as cooperative perception and hazard awareness, yet transmitting high-dimensional camera data over bandwidth-limited and dynamic wireless links remains challenging. In this paper, we propose an adaptive semantic communication architecture for vehicular multicast that adjusts the amount of visual information transmitted according to current channel conditions, available communication resources, and latency requirements. Specifically, the visual encoder uses residual quantization to represent each image at multiple bitrate and reconstruction-quality levels. At runtime, the system selects an appropriate level based on the communication conditions, enabling it to trade visual fidelity for lower transmission overhead when network resources are limited. Context-adaptive entropy coding further compresses the quantized representations using offline-learned statistics, without requiring retraining for different channel conditions. This design allows the visual representation to be learned offline while the communication strategy is adapted online. Experimental results demonstrate competitive rate-distortion performance at ultra-low bitrates, improved delivery reliability and deadline satisfaction across diverse vehicular scenarios, and strong downstream perception performance, achieving up to 0.88 mAP50 while dynamically adapting to changing communication conditions.

0.15

Defense-in-Depth at the Perception-Reasoning Interface of LLM-Centric Agentic UAV Swarms

cs.CR cs.MA ·2026-10-02 ·Mohammadhossein Homaei, Yousef Emami, Sajad Homayoun, Rahim Taheri, Hao Zhou +2 more
similarity 0.57
Abstract · 14 Pages, 7 Tables, 4 Figures

Large Language Models (LLMs) increasingly support Uncrewed Aerial Vehicle (UAV) swarm operations such as data collection scheduling, where the model reads structured sensor reports and decides which sensors to visit. An adversary who quietly manipulates those reports can redirect the swarm without modifying the model weights or the UAV. Defenses for this interface have been proposed architecturally but rarely implemented or evaluated. We implement and evaluate defense-in-depth at the perception-reasoning interface of LLM-Centric Agentic UAV Swarms. Five layers check the provenance of a report, whether its values are physically admissible, whether they agree with what swarm geometry and service history predict, whether the resulting schedule starves any sensor, and, when these fail, hand control to a deterministic scheduler that ignores the suspect input. We test each layer against an adversary strong enough to defeat the layer before it. For each of the three input-side layers, we derive in closed form how far a report can be distorted before that layer reacts, fixing each boundary from deployment parameters before any attack data is collected; across thirty matched simulation runs, predicted and measured boundaries agree. Separating attack detection from response is a well-established principle, and we quantify the cost of neglecting this distinction at the perception-reasoning interface. When the system rejects a report, it replaces it with the most recent accepted report. This prevents the adversary from controlling the UAV schedule, but it also increases cumulative cost by 79% and 74% for the two detectors, respectively, compared with the undefended system. The safety check does not detect any attacks, but it nevertheless reduces the attack-induced cost by 37.5%.

0.26

Dense Coverage, Sparse Refinement: Byte-Constrained Cooperative Perception

cs.CV ·2026-09-24 ·Melih Yazgan, Timon Müller, J. Marius Zöllner
published similarity 0.56
Abstract · Accepted at WACV 2027 (first-round acceptance)

Collaborative perception improves autonomous perception by sharing intermediate Bird's-Eye-View (BEV) features across connected agents, but dense feature exchange is difficult to deploy under strict Vehicle-to-Everything (V2X) bandwidth limits. Existing efficient methods typically either compress the full feature map uniformly, spending bits on low-value background, or sparsify communication, risking the loss of useful context. We propose a coverage-refinement design for byte-constrained cooperative perception: each agent transmits a highly compressed coarse layer over the full BEV map and allocates the remaining budget to selected high-resolution patches. A Task-Aware Benefit Selector ranks cells by estimated downstream utility, enabling deterministic budgeted refinement and zero-retraining adaptation to changing bandwidth. The receiver reconstructs a dense BEV tensor compatible with standard fusion modules. Experiments on DAIR-V2X and OPV2V show strong accuracy-payload trade-offs at kilobyte-scale budgets. On DAIR-V2X, our method reaches 0.60 AP@0.7 at only 1.87 KB per non-ego agent, compared with 0.52 at 4.61 KB for uniform SimVQ compression. Controlled diagnostics further show that the gain arises from coverage-refinement allocation rather than quantization alone. Code will be published.

0.62

Uncertainty-Driven Efficient Communication for UAV-Based Collaborative Perception

cs.CV ·2026-11-04 ·Jiuwu Hao, Yuting Wan, Liguo Sun, Zao Zhang, Yueyang Wu +1 more
published similarity 0.56
Abstract · Accepted to IEEE SMC 2026

Unmanned aerial vehicle (UAV)-based collaborative perception enables multiple agents to share complementary perceptual information, overcoming limitations such as occlusion and limited sensing range. Communication efficiency is a critical bottleneck, as UAVs operate under stringent bandwidth and computational constraints. However, existing spatial selection methods focus predominantly on foreground regions to reduce missed detections, neglecting perceptually ambiguous regions and leaving false detections unresolved. We propose UCP, an uncertainty-driven framework that characterizes collaborative value through observational uncertainty, identifying sensor-limited regions, and discriminative uncertainty, capturing model-level ambiguity near the decision boundary. These estimates drive spatial request and supply maps for efficient inter-agent communication, and the received messages are aggregated via U-Fusion, an uncertainty-guided cross-attention module. Experiments on three benchmarks across 3D object detection and trajectory prediction demonstrate a superior performance-bandwidth trade-off with high interpretability and strong generalizability.

0.24

CERF: Communication-Efficient and Retraining-Free Collaborative Perception

cs.CV ·2026-09-01 ·Jiuwu Hao, Ziyi Ni, Liguo Sun, Yuting Wan, Yueyang Wu +3 more
publishedcode/data similarity 0.56
Abstract · Accepted by ICASSP 2026

Collaborative perception shares information among multiple agents to obtain a comprehensive scene representation, enhancing the perceptual capability of individual agents. However, most existing methods rely on transmitting and fusing dense feature maps for collaboration, which incurs inevitable communication overhead and heterogeneity challenges, limiting their practicality for real-world deployment. To address these challenges, we propose CERF, a novel Communication-Efficient and Retraining-Free framework for open heterogeneous collaborative perception. In CERF, we introduce a new virtual modality (termed Poture), which is generated from the perception outputs of other agents, to augment the extracted Bird's Eye View (BEV) features of the ego agent. To mitigate transmission delays, we employ a Kalman-filter based tracker and a motion forecasting model to derive the current predictions from historical perception results. Extensive experiments demonstrate that CERF achieves performance comparable to mainstream intermediate-collaboration methods while reducing communication overhead by 95% across various downstream tasks. Furthermore, CERF enables seamless integration of unknown heterogeneous agents into the existing collaborative framework without additional retraining costs. Code is available at https://github.com/uestchjw/CERF.

0.45