CS Paper Radar

Vision-Based Leader-Follower Formation Control for Cooperative UAVs in GPS-Degraded Environments

Deekshitha Angadi, Naveena Budda, Vikas Agarwal, Rojesh Arunkumar Mulasa, Ravi Killamsetty, Mohamed Samshad, Narsimlu Kemsaram

cs.RO ·submitted 2026-09-01 15:35 UTC · v1
published

Abstract

Cooperation in multi-UAV systems requires reliable relative perception so that follower vehicles can maintain formation and continue their mission safely even when absolute positioning sensors degrade or fail. This paper presents a vision-based cooperative formation framework running on a follower UAV that uses a front-facing RGB-D camera to detect, track, and localize a leader UAV in real-time. A lightweight YOLO-based detector is trained on a dedicated drone dataset and deployed onboard to predict leader bounding boxes, which are then fused with depth information via a pinhole camera model to estimate the leader's relative pose. These estimates provide a leader-follower position controller and can also be used as a backup when GPS or external localization is unavailable. This framework is implemented as a set of ROS nodes and evaluated in a physics-based multi-UAV simulation built on XTDrone, with sensor noise and communication dropouts. We evaluate detection accuracy, runtime, and formation-keeping error under nominal conditions and under simulated failures of the positioning sensors. The results show that the proposed framework maintains stable leader-follower formations with reasonable computational cost and provides a practical basis for extending vision-based cooperative formation control to real-world multi-UAV systems.

Comment
This paper has been accepted for publication in the Proceedings of the 2026 IEEE International Conference on AI and Security for Industrial IoT Systems (AI-SIIS 2026), 24-26 September, 2026, Hyderabad, India

Closest papers

Nearest neighbours by meaning across the whole index.

PATH: Continuous Target Sensing among Autonomous Cooperative Drones

cs.RO ·2026-09-11 ·Heegyeong Kim, Alice James, Avishkar Seth, Endrowednes Kuantama, Jane Williamson +2 more
similarity 0.69
Abstract

Continuous target sensing by uncrewed aerial vehicles (UAVs) is constrained by limited flight endurance, motivating the transfer of tracking responsibility between cooperating UAVs. Such a handoff requires the receiver to identify the same physical target currently tracked by the sender despite differences in viewpoint, scale, and target appearance. Existing approaches based on global target localization or appearance-based cross-view association are limited by positioning uncertainty or ambiguous visual features. This paper presents Perspective Alignment \& Tracking Handoff (\textbf{PATH}), a platform-agnostic, geometry-assisted sensing and verification framework for target handoff between two moving UAVs. The sender reconstructs the tracked target as a metric 3D point using RGB-D sensing, while the receiver estimates its relative pose from a fiducial observation and projects the transmitted target point into its own image as a spatial prior for target acquisition. The receiver-generated candidate is then returned to the sender and verified through a cross-view Mutual Agreement Handshake before tracking responsibility is transferred. Real-world UAV experiments show mean relative-position and target-position errors of 0.047~m and 0.030~m, respectively. Under visually ambiguous conditions, PATH achieves 96.0\% frame-level receiver-side target acquisition accuracy, with 2.0\% false-positive and 2.0\% false-negative rates. A sensor-error sensitivity analysis shows that relative-pose uncertainty is the dominant contributor to receiver-view projection error. The implementation operates at video rate with compact inter-UAV communication below 16~kB/s at 60~Hz, demonstrating the feasibility of lightweight geometry-assisted target handoff on resource-constrained UAV platforms.

0.37

Reliability-Adaptive Multimodal Relative Pose Tracking for Underwater Leader-Follower Robots

cs.RO ·2026-11-09 ·Mingyang Yang, Zeyu Sha, Xiaorui Wang, Jun Wang, David Chen Wang +1 more
code/data similarity 0.67
Abstract

Underwater leader-follower formation control requires continuous relative-pose estimation despite degraded sensing conditions and limited onboard computation. This paper presents a reliability-adaptive multimodal perception framework for six-degree-of-freedom relative-pose tracking between two moving underwater robots. The framework integrates complementary visual, acoustic-ranging, and hydrodynamic measurements while accounting for their distinct physical characteristics and time-varying reliability. It adaptively weights modality contributions and models temporal dependencies with linear computational complexity, enabling efficient pose estimation on embedded hardware. Extensive experiments were conducted using a custom-built AUV platform, with 648 pool trials spanning two leader vehicle types, three illumination conditions, and nine trajectory patterns. Evaluation yields an average translational error of 3.8 cm and an average orientation error of 2.4°. On an onboard NVIDIA Jetson AGX Orin, the fusion network achieves an inference latency of 17.62 ms, providing a 2.69-fold speedup over the evaluated Transformer-based fusion baseline. Closed-loop experiments further demonstrate stable formation maintenance with a moving leader under onboard computational constraints. A multimodal dataset containing measurements from all 648 trials will be made publicly available.

0.28

Borrowed Eyes: Markerless Nano-UAV Flight with an Active Quadruped Observer

cs.RO ·2026-10-07 ·Alejandro Lorite Mora, Dimitrios Arapis, Andrés Faíña
similarity 0.66
Abstract · 8 pages, 7 figures, 2 tables. Submitted to ICRA 2027

Nano unmanned aerial vehicles (nano-UAVs) can navigate confined spaces that larger robots cannot, but their payload capacity severely limits the sensors and compute available for self-localization in global navigation satellite system (GNSS)-denied environments. We present a vision-based system that localizes a nano-UAV from a quadruped robot with an arm-mounted camera. The quadruped tracks the drone, estimates its position in its own coordinate system using segmentation masks and depth, and transmits that position over a real-time radio link. To ensure continuous tracking, we developed a perception-aware nonlinear Model Predictive Controller (NMPC) that dynamically adjusts the quadruped's body and arm to maximize the drone's visibility, treating observation reliability as a primary control objective. Relying solely on this external estimate, the nano-UAV executed predefined trajectories from takeoff to landing with a median 3D localization error of 59 mm. The system demonstrates robustness against short visual occlusions by smoothly transitioning to onboard inertial flight when line-of-sight is temporarily lost. Ultimately, this framework allows a quadruped to offload the localization burden of a nano-UAV, enabling inspection of complex spaces that neither robot could navigate alone.

0.20

JitTrack: Onboard Multi-Object Tracking Against Viewpoint Jitter for Agile UAVs

cs.RO ·2026-08-11 ·Yachun Shan, Feitian Zhang
similarity 0.65
Abstract · 9 pages, 7 figures. Submitted to IEEE Robotics and Automation Letters

Multi-object tracking (MOT) onboard agile unmanned aerial vehicles (UAVs) remains challenging due to severe viewpoint jitter induced by camera ego-motion. Rapid attitude changes during flight often lead to significant target displacement across frames, causing inaccurate target association and degraded tracking performance. Existing UAV MOT methods are primarily evaluated on offline benchmarks and seldom address the practical requirements of real-world onboard deployment, including robustness to camera motion and active target following. To address these challenges, we propose JitTrack, an active onboard multi-object tracking framework that accommodates drone dynamics and camera ego-motion. Built upon a query-based transformer tracker, JitTrack introduces semantic refinement to improve the detection of emerging targets, motion-aware query rectification to compensate for target misalignment caused by viewpoint jitter, and a motion-inspired denoising training strategy that simulates camera motion patterns for robust supervision. Furthermore, we develop a perception-planning-control closed-loop tracking pipeline for real-world deployment, enabling collision-free and physically feasible target following on agile UAVs. Extensive experiments on public UAV MOT benchmarks demonstrate consistent improvements over the baseline method, while real-world flight experiments validate the effectiveness and practicality of JitTrack for robust onboard visual tracking under viewpoint jitter.

0.12

A Heterogeneous UAV UGV Methodology for Aerial Tracking of Mini Autonomous Cars

cs.RO ·2026-09-21 ·Santiago Fernandez, Angel da Silva, Hiago Sodre, Pablo Moraes, Bruna Gutterres +1 more
published similarity 0.64
Abstract · This paper was accepted at the Latin American Robotics Symposium LARS 2026

Aerial tracking of autonomous ground vehicles can provide useful situational awareness in environments where fixed external perception infrastructure is unavailable. However, visual-only tracking from an unmanned aerial vehicle (UAV) is limited by camera field of view, viewpoint changes, occlusions, and temporary loss of target visibility. This paper presents a heterogeneous UAV--UGV methodology for aerial tracking of small-scale autonomous ground vehicles in simulation. The system is implemented in ROS~2 and Gazebo Sim using a Tello-like UAV equipped with a front-facing RGB camera and multiple LIMO-like unmanned ground vehicles (UGVs). Three approaches are evaluated: a training-free color-marker baseline, a YOLO-based visual baseline, and a proposed UAV--UGV fusion method. The proposed method uses the same visual detector as the YOLO baseline, but combines visual detections with UGV state information to maintain the target estimate during visual gaps. The methods are evaluated across four scenarios: single-car tracking, crossing trajectories, occlusion, and UAV-assisted awareness. Results over 600 simulation runs show that the proposed fusion method achieves nearly continuous track coverage across all scenarios while reducing localization error relative to the YOLO baseline. Compared with the visual-only YOLO approach, the fusion method reduces RMSE by 37.3\%, 64.2\%, 62.6\%, and 61.8\% in the four evaluated scenarios, respectively. The method also enables conflict-awareness lead times of 10.20~s in the crossing scenario and 9.07~s in the UAV-assisted awareness scenario. These results indicate that heterogeneous UAV--UGV fusion can improve aerial tracking continuity and robustness when visual observations are intermittent.

0.22

EVPeriscope: Extended Perception across Aerial and Ground Vehicles with Event-based Propeller Tracking

cs.RO ·2026-09-10 ·Dexter Ong, Vijay Kumar, Pratik Chaudhari
code/data similarity 0.62
Abstract

Reliable relative localization between aerial and ground robots is a key requirement for tightly coordinated heterogeneous teams. This can be difficult to do using conventional frame-based cameras and fiducial markers because they are sensitive to motion blur, lighting variations, and payload constraints. This paper presents EVPeriscope, an event-based perception system that enables detection, localization and control of a quadrotor using an upward-facing event camera on a ground robot by detecting the high-frequency visual signature of its propellers. This system allows the quadrotor to function as an extended perception system for the ground robot when onboard sensors exhibit degradation or occlusion. We demonstrate the capabilities of this marsupial ground-aerial system via experiments in challenging field conditions with wind speeds of up to 15 mph, in both daylight and at night. We show that the system supports localization and closed-loop navigation through dense foliage where the ground robot's sensors are occluded. Our control system for the quadrotor operates at 200 Hz entirely with onboard sensing and computation. More details and experiment videos can be found on the project page: https://ongdexter.github.io/evperiscope.

0.35

Towards Agile Vision-Based Multi-UAV Flight: Revisiting State Estimation

cs.RO ·2026-09-30 ·Michal Pliska, Matouš Vrba, Ondřej Víta, Martin Jiroušek, Viktor Walter +1 more
publishedbenchmark similarity 0.62
Abstract · This work has been accepted to the IEEE for possible publication (IROS 2026)

Agile multi-UAV flight requires accurate and low-latency onboard estimation of the kinematic states of neighboring UAVs for collision avoidance, motion coordination, etc. Most vision-based approaches rely on position-only measurements, inferring velocity and acceleration indirectly from displacement. We show that this introduces a fixed structural delay in the estimation of higher-order states, which limits the achievable agility. To address this, we propose to integrate tilt measurements, provided by a state-of-the-art visual detector, which inform about the thrust direction of co-planar multirotor UAVs. We benchmark four position-only and five pose-aware estimators, including a novel formulation of a linear thrust-constraining Kalman filter, on two real-world and one high-fidelity photorealistic simulated dataset over different levels of agility (3-21 m/s^2). In our setup, pose-aware estimation consistently reduces the average velocity and acceleration estimation errors by 40% and 57% across the three datasets with the proposed KF formulation outperforming the other estimators. Position-only filters exhibit a constant ~300 ms delay in acceleration step response independent of agility, whereas the tilt-constrained estimators operate near the physical response limit given by the camera frame-rate by observing the change in thrust direction before the displacement accumulates. In a closed-loop leader-follower simulated experiment with NMPC control, position-only estimation of the leader's state fails to facilitate stable hovering of the follower, while the proposed estimator enables tracking of lateral maneuvers exceeding 2g of acceleration.

0.26

SwarmNxt: Open-source Software-Hardware Platform for Fast and Agile Aerial Swarms

cs.RO ·2026-09-10 ·Charbel Toumieh, Niel Mistry, Benjamin Jarvis, Simon Jeger, Peize Liu +2 more
code/data similarity 0.62
Abstract

Aerial robot swarms have the potential to transform time-critical safety, security, and search-and-rescue operations. By coordinating multiple robots, they can rapidly survey disaster sites, map collapsed or GPS-denied environments, and search cluttered areas faster than a single robot, reducing response times and minimizing risks to first responders. Realizing this potential, however, requires robust autonomous swarm navigation, which remains an active research challenge. Progress is further constrained by existing platforms, as commercial drones are often closed-source or lack the onboard computational resources needed for agile, vision-based collective flight. Moreover, developing, deploying, and maintaining software across multiple aerial robots requires significant engineering effort. To address these challenges, we present SwarmNxt, an open-source software platform built on the open-source OmniNxt drone hardware. SwarmNxt provides an end-to-end toolkit, including detailed hardware assembly instructions with a video tutorial, automation tools for parallel software deployment and swarm-wide updates, and a ROS 2-based framework for autonomous navigation. The platform integrates state-of-the-art control, planning, and depth estimation into a single ROS 2 multi-agent system, providing an open research infrastructure for physical swarm experimentation. We validate SwarmNxt through two real-world experiments: a six-drone swarm performing decentralized planning with high-speed inter-drone collision avoidance, and a four-drone swarm executing collective flight with onboard depth estimation in an obstacle-filled environment. Both experiments were run indoors with global position from external motion capture; perception, planning, and control run onboard.

0.33