CS Paper Radar

Reinforcement Learning-Based 3D Beam Adaptation for Underwater Wireless Optical Communication with AUVs

Viviana Centritto, Ama Bandara, Sergi Abadal, Evgenii Vinogradov

eess.SP cs.ETeess.SY ·submitted 2026-10-05 14:25 UTC · v2

Abstract

Underwater Wireless Optical Communications (UWOC) provide essential high data rates for Autonomous Underwater Vehicles (AUVs), but reliable connectivity is critically affected by transmitter-receiver misalignment. This work addresses the beam pointing problem for a moving AUV subject to unknown ocean currents through a Deep Reinforcement Learning (DRL) framework. We develop a comprehensive 3D UWOC channel model incorporating depth-dependent attenuation, turbulence, and a discrete-ray method to accurately quantify geometric and misalignment losses. The resulting agent jointly optimizes beam steering and divergence angles, learning a policy that prioritizes continuous link maintenance. Evaluated using real oceanographic data, the framework's performance is assessed via the excess outage metric, which isolates outages occurring exclusively due to pointing errors. Relative to perfect alignment between nodes, the proposed method bounds this metric to 8.5% under ocean current-free tracking conditions and limits it to 16.2% when subjected to current-induced drift. The findings obtained in this work confirms that DRL-enabled beam control is capable of adaptive tracking and mitigating link outages in dynamic underwater environments.

Comment
6 pages, in IEEE International Conference on Wireless for Space and Extreme Environments (WiSEE), workshop on Optical Wireless Communications and Sensing in Sea, Air, and Space (OWC-SAS)

Closest papers

Nearest neighbours by meaning across the whole index.

Multi-AUV Ad-hoc network-based Target Tracking: A Value Gradient Guidance Multi-Agent Diffusion Reinforcement Learning Approach

cs.LG cs.MAcs.NI ·2026-08-12 ·Jiaao Ma, Chuan Lin, Guangjie Han, Shengchao Zhu, Qian Zhu +2 more
similarity 0.67
Abstract

Multi-AUV ad-hoc network-based target tracking requires networked autonomous underwater vehicles (AUVs) to cooperatively track maneuvering targets under constrained acoustic communication, dynamic topology, and uncertain ocean disturbances. Although multi-agent reinforcement learning (MARL) enables decentralized coordination through centralized training, existing methods suffer from high-dimensional joint state-action modeling, noise-sensitive policy generation, leading to unstable training and degraded tracking. To address these issues, we propose VGG-MADiffRL, a value-gradient-guided multi-agent diffusion RL algorithm, and MDCA, a diffusion?based hierarchical control architecture. Leveraging underwater mission characteristics, we model sonar detection mechanisms and ocean current disturbances, formulating cooperative tracking for multi-AUV ad-hoc networks as an MDP. The proposed MDCA constitutes a three-tier closed-loop control framework: a global intelligent control layer, a local online training layer, and a physical action execution layer. This structure enables synergistic optimization across task allocation, local decision processes, and execution feedback. Within MDCA, the local online training layer is the policy learning framework; VGG-MADiffRL builds on diffusion policies and incorporates value gradients to guide action generation in the reverse denoising process, steering the generated actions towards higher expected returns. It employs twin value networks with joint optimization and soft target updates to mitigate overestimation and training oscillations, promoting more stable convergence. Experimental results show that VGG-MADiffRL consistently achieves faster convergence, higher tracking accuracy, and smoother training dynamics in cooperative tracking scenarios, validating its effectiveness and practical engineering value in dynamic underwater settings.

0.04

NAViLoss: An Underwater Navigation-Aware Dual-Residual Objective for Physics-Consistent Learning

cs.RO cs.LG ·2026-10-07 ·Arup Kumar Sahoo, Itzik Klein
similarity 0.67
Abstract · 26 pages, 7 figures

Autonomous underwater vehicles (AUVs) commonly rely on inertial navigation systems (INS) aided by Doppler velocity logs (DVLs) for reliable underwater navigation. Accurate DVL velocity estimation is therefore essential for successful operation. Recent learning-based methods have demonstrated improved DVL velocity estimation, particularly under degraded measurement conditions. However, their training objectives typically rely on conventional regression losses that are highly sensitive to large residuals and corrupted observations. Additionally, they do not explicitly account for the physical consistency and measurement uncertainty associated with the underlying sensing process. To address these limitations, this paper introduces navigation-aware loss (NAViLoss), a robust and uncertainty-aware objective function for learning-based AUV velocity estimation. NAViLoss jointly penalizes the velocity-estimation residual in the navigation-state domain and the beam-consistency residual in the DVL measurement domain. Its bounded formulation limits the influence of large residuals, while an adaptive mechanism regulates the uncertainty in beam geometry. Furthermore, NAViLoss is integrated with a DeepONet architecture to form a novel NAVi-DeepONet model for seamless estimation of an underwater vehicle's velocity. Lastly, our model is evaluated using approximately 10,000m of semi-synthetic AUV experimental data collected during multiple real-world sea trials. Experimental results demonstrate a 44% improvement in velocity-estimation accuracy compared with conventional and learning-based baselines. These results demonstrate the effectiveness of navigation-aware and uncertainty-adaptive loss design for robust learning-based underwater velocity estimation.

0.15

Machine Learning for Underwater Optical Wireless Communication Systems: A Comprehensive Survey

eess.SP cs.LG ·2026-09-14 ·Shaymaa Mahmoud, Ardimas Purwita, Mohamed-Slim Alouini
survey similarity 0.65
Abstract

Underwater Optical Wireless Communication (UOWC) has emerged as a promising technology for high-speed underwater data transmission, offering significantly higher bandwidth and lower latency compared to acoustic and Radio Frequency (RF) technologies. However, the underwater medium introduces severe impairments that degrade link performance and limit communication range. The growing complexity of these challenges has increased research interest in Machine Learning (ML) and Deep Learning (DL) approaches, which offer powerful tools for channel modeling, signal processing, and system adaptation in ways that conventional analytical methods struggle to achieve. This survey provides a comprehensive review of ML methods applied across the full UOWC system pipeline, covering channel modeling, transmitter design, receiver detection and equalization, link alignment, and emerging applications, including wireless power transfer, semantic communication, object detection, optical sensing, and localization. Finally, we discuss open challenges and future research directions to motivate further work in this rapidly growing field.

0.21

AquaOrbit: Sim-to-Real Reinforcement Learning for Underwater Target Orbiting under Intermittent Visual Feedback

cs.RO ·2026-09-21 ·Kanzhong Yao, Jinyi Leng, Hao Zhang, Zhe Sun, Xuelong Li
similarity 0.60
Abstract · This work has been submitted to IEEE for possible publication

Intermittent visual loss disrupts target-relative feedback during underwater orbiting, making it difficult to maintain coordinated motion and reacquire a moving target. We present AquaOrbit, a reinforcement-learning controller with a recovery module for underwater target orbiting under interrupted visual feedback. During detection loss, the recovery module uses latched line-of-sight, roll, and depth references to support stabilization and target reacquisition. We train the controller in Isaac Sim with dynamics, observation, and vision-loss randomization. Evaluated without retraining in Gazebo/ROS2 under a different physics engine and perception perturbations, AquaOrbit completes 20/20 orbiting trials in each of the static- and moving-target conditions on an unseen variable-depth 3-D trajectory. In the moving-target condition, it reduces mean line-of-sight error by approximately 46% relative to a PID-based visual servoing controller with recovery while maintaining comparable path-tracking accuracy; removing the recovery module reduces completion to 9/20. Zero-shot physical deployment with fully onboard perception and control demonstrates elliptical, figure-eight, and variable-depth circular trajectories, including the latter two trajectory types absent from training. The robot maintains attitude stability during manual occlusions lasting up to 8s and reacquires the target within 2.5s in the reported attitude-induced field-of-view loss events.

0.24

FinsSim: A Reality-Aligned Integrated Simulation Platform for Underwater Robot Learning

cs.RO eess.SY ·2026-09-20 ·Yu Zhang, Yuanmingqing Song, Xiangyun Rao, Pangkit Fong, Kunhao Zhang +2 more
similarity 0.59
Abstract · 8 pages, 6 figures

Underwater robot learning relies on simulators that integrate high-fidelity hydrodynamics, convenient learning interfaces, and a credible transition to real scenarios. In this work, we present FinsSim, a reality-aligned integrated simulation platform for Sim-to-Real underwater robot learning. FinsSim first constructs high-fidelity simulation with selectable backends to adapt to diverse requirements. To facilitate underwater robot research, it further offers standard control baselines, alongside with unified robot learning workflows. For reliable Sim-to-Real transfer, FinsSim adopts a multi-sensor fusion scheme to provide low-cost yet precise localization. Moreover, it implements calibrated thruster-hydrodynamics models and a constrained wrench allocation algorithm. Bridging these modules by ROS~2, FinsSim establishes a complete Sim-to-Real transfer pipeline. Through matched simulations and experiments, it is demonstrated that reliable Sim-to-Real transfer of underwater robot control policies can be achieved with the FinsSim framework. Separate ablation studies also validate that the modules of FinsSim can address the pivotal issues of underwater Sim-to-Real from different aspects. Overall, this work aims to bridge the gap between theoretical research and practical applications, ultimately driving advancements in the field of underwater robotics.

0.27

Conditional Diffusion Generation of DVL and Inertial Signals for Underwater Navigation

cs.LG ·2026-10-25 ·Michal Levin, Itzik Klein
similarity 0.59
Abstract

Autonomous underwater vehicles (AUVs) navigate without satellite positioning and rely on Doppler velocity log (DVL) and inertial sensors for motion estimation. Deep learning methods for processing these sensors require training data covering the motion the AUV encounters, commonly obtained through costly sea trials. To overcome this limitation, we present a conditional diffusion framework that generates synthetic DVL and inertial signals to replace real recordings in training. The generator employs an Elucidated diffusion model conditioned on three per-channel motion descriptors. Three configurations are evaluated: DVL beams alone, inertial channels alone, and all channels jointly from one model. Across thirteen trajectories of the real dataset, the generated signals followed the requested conditions with Pearson correlations between 0.72 and 0.997, transferred to trajectories withheld from generator training, and did not replicate any training window. A BeamsNet velocity estimator trained on generated signals alone, with no real recording retained, matched the velocity error obtained with the full real training set. Replacing up to all real windows with generated ones caused no measurable loss. Thus, this approach reduces the dependence of learned AUV navigation on sea-trial data, enables training when recordings are scarce, and facilitates developing and evaluating deep learning methods for underwater navigation.

0.18

Deep Reinforcement Learning for Blind Passive Beamforming in RIS-Assisted MIMO Systems

cs.IT stat.ML ·2026-10-29 ·Oumaïma Bounhar, Thomas Chêne, Ghaya Rekaya-Ben Othman, Oussama Damen
published similarity 0.57
Abstract · Accepted at AIxNET 2026. 6 pages. Copyright may be transferred to IEEE after publication, after which this version may no longer be accessib…

Reconfigurable Intelligent Surfaces (RIS) are a promising solution for improving millimeter-wave (mmWave) communications by controlling signal propagation. Yet, selecting the optimal RIS configuration remains challenging due to the high-dimensional search space and signaling overhead. We propose a Deep Reinforcement Learning (DRL) framework for blind passive beamforming, where a Deep Q-Learning (DQL) agent learns an adaptive policy to select near-optimal RIS configurations while sounding only few codewords, without channel state information. Results show that our approach outperforms Q-Learning (QL) and conventional baselines in convergence, performance, and overhead reduction.

0.34

Dynamical System-Based Imitation Learning and Neuroadaptive Control for Trajectory Recovery in Autonomous Ships

eess.SY cs.RO ·2026-08-25 ·Yeyson A. Becerra-Mora, José Ángel Acosta
similarity 0.57
Abstract · Preprint submitted to journal (under review). 22 pages, 8 figures, 3 tables

Repetitive maritime operations can be effectively learned using the Imitation Learning (IL) paradigm, which transfers human expertise directly to Unmanned Surface Vehicle (USV) control systems. Dynamical Systems (DS) are widely used to model non-linear human demonstrations while offering inherent stability guarantees. However, real-world execution under persistent marine perturbations reveals a critical trade-off: standard DS-based IL approaches prioritize global target convergence at the expense of localized trajectory reproduction fidelity. To address this limitation, we present a hybrid learning-control architecture that integrates a DS-based IL reference generator with a neuroadaptive controller. Our approach introduces a control action that drives the USV back to the demonstrated path following exogenous disturbances, enabling dynamic human-like reactive alignment-termed behavioral tracking. The proposed methodology is validated using the Marine Systems Simulator (MSS) toolbox. Simulation results confirm that the framework generalizes complex maneuvering tasks while substantially improving trajectory tracking fidelity under disturbances compared to alternative control strategies.

0.12