CS Paper Radar

Deep Learning for Atmospheric Turbulence Mitigation: A Comparison Between GANs and Transformers

Juan S. Carrillo, Jérôme Gilles

eess.IV cs.CV ·submitted 2026-10-13 22:50 UTC · v1
publishedhot direction

Abstract

Atmospheric turbulence creates complex spatial and temporal distortions that significantly reduce image quality. This work compares two deep learning methods for turbulence reduction: Generative Adversarial Networks (GANs) and Transformer-based models. Both methods are trained and tested on the SOTIS dataset, which includes multi-frame sequences under different turbulence conditions with ground truth references. Our results show an important finding: GAN-based models perform better in severely degraded, low-quality situations, while Transformer-based models achieve better results in moderate to high-quality settings, especially in correcting geometric distortions. We also enhance the Turbulence Mitigation Transformer (TMT) by introducing a refined training strategy, an augmented input design that includes a temporally averaged frame as prior structural information, and a modified loss function that incorporates the Structural Similarity Index (SSIM). These improvements lead to consistent performance gains across various turbulence conditions. Overall, this study provides an in-depth evaluation of deep learning methods for turbulence reduction and offers practical guidance for selecting models in different imaging environments.

Comment
Published in the Journal of Electronic Imaging
Journal ref
J. S. Carrillo and J. Gilles, "Deep learning for atmospheric turbulence mitigation: A comparison between GANs and Transformers," Journal of Electronic Imaging, 35(5), 053014 (2026)
DOI
10.1117/1.JEI.35.5.053014

Closest papers

Nearest neighbours by meaning across the whole index.

GenONet: A Generative operator Network for High-Resolution Precipitation Nowcasting

cs.LG physics.ao-ph ·2026-09-01 ·Mohammad Kian Golkar, Luciano Alves de Oliveira, Mohammad Khanjani
similarity 0.66
Abstract · 28 pages, 11 Figures

High-resolution precipitation nowcasting is critical for reducing the impacts of severe weather but remains difficult because of rapid storm evolution. Deep learning models have shown great promise for this task, but their predictive skill often deteriorates over longer forecast horizons. This leads to increasingly blurry forecasts that fail to capture the complex, non-linear evolution of storm systems. In order to address these limitations, we introduce Spatio-Temporal U-DeepONet (GenONet), a novel architecture for long-range precipitation forecasting up to 3 hours, specifically designed to produce sharp and physically consistent results. GenONet's architecture pioneers the use of a Deep Operator Network (DeepONet) as a generator within a Generative Adversarial Network (GAN) framework for this task. The DeepONet learns the continuous-time dynamics of precipitation, ensuring stability over long forecast horizons. Adversial training against a spatio-temporal discriminator compels the model to produce sharp, coherent forecasts, while a physics-informed loss regularizer, derived from the Moisture Conservation Equation, improves physical plausibility in our ablation setting. Quantitative evaluations show that our model achieves consistently higher scores on most of the metrics, especially for highintensity events and at longer lead times. Qualitatively, GenONet produces structurally coherent forecasts that maintain their integrity, whereas baseline models degrade into indistinct patterns. Finally, an ablation study confirms the benefit of this physics-informed loss, highlighting the strength of combining operator learning with adversarial training.

0.29

TriINR: Unsupervised Atmospheric Turbulence Mitigation with Triple Implicit Neural Representations

cs.CV ·2026-10-16 ·Yuka Ogino, Rikuto Otsuka, Yuho Shoji, Takuya Ogawa, Takahiro Toizumi +1 more
published similarity 0.66
Abstract · Accepted to WACV2027

Atmospheric turbulence degrades image quality in long-range imaging. The degradation is generally modeled with two spatio-temporally varying components, tilt and blur. Recent unsupervised turbulence mitigation methods have demonstrated the effectiveness of representing a sharp image as an implicit neural representation (INR). However, their tilt and blur representations remain frame-dependent, and the number of optimization parameters grows with the number of input frames. To address this limitation, we propose TriINR, an unsupervised turbulence mitigation method that recovers a sharp image from multiple degraded frames. TriINR represents the sharp image, tilt, and blur as three implicit neural representations. By modeling tilt and blur as spatio-temporal INRs, TriINR achieves frame-count-independent degradation representation and memory-efficient optimization. We further incorporate physical and statistical properties of atmospheric turbulence as constraints on the INRs to improve restoration accuracy. Experiments on synthesized turbulence data with three degradation strengths show that TriINR restores images more robustly across varying turbulence strengths than prior TM methods and achieves the highest accuracy among the evaluated unsupervised methods. Our method also demonstrates robustness on real long-range scenes containing additional non-ideal motion.

0.44

Tropical Cyclone Wind Field Diagnosis with Image-to-Image Models

cs.CV physics.ao-ph ·2026-10-07 ·Sarah J. Ollier, David S. Nolan, Brian D. McNoldy, Eduardo Siman
similarity 0.63
Abstract · Preprint prior to peer review. Submitted to Artificial Intelligence for the Earth Systems on 8th September 2026

This study presents a novel application of image-to-image generative adversarial networks (GANs) to produce a complete tropical cyclone (TC) surface wind field diagnosis from a single frame of satellite infrared brightness temperature (IR). Because TC surface wind fields are infrequently observed, we train on two data sets. The first contains 3,323 image pairs of IR and wind fields from Weather Research and Forecasting Model (WRF) simulations (WRF2WRF). The second contains 11,886 real IR satellite images paired with parametric wind fields built on the extended best-track data set (IBTrACS) of TC intensities and wind radii (GOES2PARA). Both models, conditioned on storm intensity, motion, and latitude, transform single-channel IR into wind fields over 6$^{\circ} \times$ 6$^{\circ}$ (WRF2WRF) and 7$^{\circ} \times$ 7$^{\circ}$ (GOES2PARA) domains, at 256 $\times$ 256 pixel resolution. GOES2PARA captures outer wind field asymmetries, enabling diagnosis of 64, 50, and 34-knot wind radii (R64, R50, R34) in each quadrant. WRF2WRF also produces detailed rainbands and eyewall asymmetries. Pixel-wise RMSE against held out truth demonstrates realistic wind field generation with mean RMSE of 6.4 knots (GOES2PARA) and 7.2 knots (WRF2WRF). Compared against IBTrACS, WRF2WRF achieved mean absolute errors (MAE) of 14.8 nautical miles (NM) for radius of maximum wind (RMW) and 25.9 NM for R34. GOES2PARA achieved 10.0 NM MAE for RMW and 25.9 NM for R34. Against Synthetic Aperture Radar wind fields GOES2PARA yielded a lower mean pixel-wise error (RMSE 12.1 kt, MAE 9.5 kt) than WRF2WRF (RMSE 13.9 kt, MAE 11.3 kt). Unlike many existing operational surface wind speed algorithms, our image-to-image models provide updated wind fields with every new satellite image.

0.22

Extreme-Scale Super-Resolution of Homogeneous Turbulence from a Single Realization

cs.LG physics.flu-dyn ·2026-10-14 ·Arth Sojitra, Muhammad Abid, Omer San
similarity 0.63
Abstract

Homogeneous turbulent flows exhibit recurring multiscale structures across spatially separated regions, suggesting that a single flow realization may contain sufficient information to learn representations of unresolved scales. We investigate this hypothesis through an extreme super-resolution framework in which a wide-activation deep super-resolution (WDSR) network is trained exclusively on overlapping patches extracted from a single high-resolution turbulent realization. Previously unseen flow fields are reconstructed through patchwise prediction followed by Hann-window-weighted overlap-add assembly. The framework is evaluated for two-dimensional homogeneous isotropic turbulence at super-resolution factors of x32 and x64, representing severe spatial under-resolution. Despite the limited training data, the proposed approach reconstructs coherent vortical structures and recovers fine-scale features more accurately than conventional interpolation. Spectral and statistical analyses further demonstrate improved recovery of high-wavenumber energy content and multiscale flow characteristics. To assess the effectiveness of exploiting internal spatial information, we compare the proposed framework against WDSR models trained on 1, 25, 50, and 100 complete high-resolution snapshots without patch extraction. This comparison distinguishes the benefits of localized learning from those associated with increasing the number of training realizations. The results demonstrate that exploiting spatial redundancy within a single turbulent realization enables accurate reconstruction of previously unseen flow fields, even under extreme spatial under-resolution. These findings establish single-realization patch-based learning as a promising data-efficient approach for turbulent super-resolution when high-resolution training data are scarce.

0.22

A high-dimensional spatial-physical coupling operator for generalizable super-resolution of three-dimensional turbulent flows

cs.CV physics.flu-dyn ·2026-11-17 ·Zengyuan Zuo, Junjun Jiang, Fei Mu, Xubo Wang, Xianming Liu
similarity 0.63
Abstract

Accurate representation of turbulent flows requires simultaneously capturing multi-scale spatial structures and the complex, nonlinear coupling between distinct physical quantities (\textit{e.g.}, velocity components and density). In practice, however, turbulent data are often constrained by low spatial resolution due to hardware limitations and computational costs. While current data-driven super-resolution methods have shown promise by learning mappings via convolutional neural networks (CNNs), they typically treat physical variables as independent channels. This simplified approach neglects intrinsic cross-variable correlations, causing models to struggle with high-fidelity reconstruction, particularly under extremely sparse conditions or in out-of-distribution regimes. To bridge this gap, we propose an architecture-agnostic spatial-physical coupling operator that extends the standard convolution into a joint spatial-variable dimension. Unlike conventional linear channel mixing, our framework constructs a unified spatial-physical representation within CNNs to explicitly capture local inter-variable interactions. By lifting features into a higher-dimensional manifold, this operator significantly enhances the model's representation capacity. Experiments demonstrate that this approach enables strong reconstruction performance and zero-shot generalization, offering a physics-aware inductive bias for super-resolution of three-dimensional turbulence. More broadly, this work suggests that high-dimensional operators offer a promising avenue for modeling intricate multi-scale turbulent flows.

0.11

Synthetic Data Augmentation for Satellite-Based Analysis of Battle-Damaged Agricultural Fields in Ukraine

cs.CV cs.AI ·2026-08-17 ·Marta Sumyk, Oleksandr Kosovan, Iryna Voitsitska
similarity 0.62
Abstract

Monitoring war-induced damage to agricultural land in Ukraine is important for understanding threats to food security, environmental stability, and post-war recovery. However, the development of computer-vision systems for satellite-based damage analysis is limited by the scarcity of labeled imagery, especially for damaged agricultural fields. This work investigates synthetic data augmentation as a method for improving classification under limited and imbalanced training data. We train class-conditional Generative Adversarial Network (GAN) and Denoising Diffusion Probabilistic Model (DDPM) architectures on real satellite images and use them to generate additional bombed and not-bombed agricultural-field samples. The generated images are used only for training augmentation, while all downstream evaluation is performed on an exclusively real test set. A Vision Transformer classifier is trained under multiple real and synthetic data configurations to measure the practical utility of each generative approach. The best configuration, based on balanced DDPM augmentation, improves accuracy from 84\% to 88\%, balanced accuracy from 67\% to 81\%, macro F1 from 65\% to 78\%, and recall for the underrepresented not-bombed class from 41\% to 69\%. These results demonstrate the potential of synthetic satellite imagery for data-scarce geospatial applications in war-affected regions.

0.19

Generative AI in Turbulence: Foundations, Applications, Validations and Benchmarks

cs.AI physics.flu-dyncond-mat.stat-mechcs.LG+1 ·2026-10-26 ·Luca Biferale, Michele Buzzicotti
benchmarkposition similarity 0.62
Abstract · submitted to JFM-Perspective

Generative AI offers new approaches to turbulence problems in which incomplete observations or unresolved d.o.f. admit multiple physically plausible solutions. Potential applications include stochastic closures, super-resolution, data assimilation, uncertainty quantification, and the generation of Eulerian and Lagrangian flow ensembles. However, turbulence is not a conventional image-generation problem. Its multiscale dynamics, intermittency, extreme events, rough velocity fields and intrinsically stochastic transport make visual realism, pointwise error, and low-order spectral agreement insufficient measures of physical fidelity. We examine where generative AI provide novel capabilities with emphasis on diffusion models, score-based methods, and related probability-transport formulations. We discuss their use in unconditional generation, probabilistic reconstruction, reduced-order modeling, temporal evolution, and mesh-free representations. We argue that validation must address turbulence properties: energy transfer, structure functions, anomalous scaling, pdf tails, coherent structures, conservation laws, temporal consistency, ensemble diversity and generalization. Finally, we discuss potential limitations in validation, baseline, data availability and reproducibility, and the need for a benchmark ecosystem combining open data, standardized challenges, hidden tests, physics-aware metrics, and documented failures. This paper serves as a bridge between two communities. For AI practitioners, we provide a fair and accessible introduction to turbulence, essential for appreciating the challenges posed by the field and for assessing the physical relevance of generative models. For turbulent practitioners, we provide an introduction to the key technological aspects of Generative AI, clarifying what these tools actually do, and where their potential and current limitations lie.

0.10

sRGB Real Noise Modeling via Noise-Aware Sampling with Normalizing Flows

cs.CV ·2026-08-29 ·Dongjin Kim, Donggoo Jung, Sungyong Baik, Tae Hyun Kim
publishedbenchmarkhighly cited similarity 0.62
Abstract · ICLR 2024

Noise poses a widespread challenge in signal processing, particularly when it comes to denoising images. Although convolutional neural networks (CNNs) have exhibited remarkable success in this field, they are predicated upon the belief that noise follows established distributions, which restricts their practicality when dealing with real-world noise. To overcome this limitation, several efforts have been taken to collect noisy image datasets from the real world. Generative methods, employing techniques such as generative adversarial networks (GANs) and normalizing flows (NFs), have emerged as a solution for generating realistic noisy images. Recent works model noise using camera metadata, however requiring metadata even for sampling phase. In contrast, in this work, we aim to estimate the underlying camera settings, enabling us to improve noise modeling and generate diverse noise distributions. To this end, we introduce a new NF framework that allows us to both classify noise based on camera settings and generate various noisy images. Through experimental results, our model demonstrates exceptional noise quality and leads in denoising performance on benchmark datasets.

0.54