CS Paper Radar

Vision Transformers for Estimating Irradiance Using Data Scarce Sky Images

David Hamlyn, Sunny Chaudhary, Tasmiat Rahman

eess.IV cs.AIcs.CVcs.LG ·submitted 2026-10-22 08:57 UTC · v1
publishedhighly cited

Abstract

Accurate estimation of diffuse horizontal irradiance (DHI) is critical for optimising photovoltaic system performance and energy forecasting yet remains challenging in regions lacking comprehensive ground-based instrumentation. Recent advancements using Vision Transformers (ViTs) trained on extensive sky image datasets have shown promise in replacing costly irradiance measurement equipment, but the scarcity of long-term, high-quality sky imagery significantly restricts practical implementation. Addressing this critical gap, this study proposes a novel dual-framework approach designed for data-scarce scenarios. First, calculated atmospheric parameters, including extraterrestrial irradiance and cyclic time encodings, are integrated to represent sky conditions without utilising any instrumentation. Next, a sequential pipeline initially predicts synthetic global horizontal irradiance (GHI) and uses it as a feature, to refine DHI estimation. Finally, a dual-parallel architecture simultaneously processes raw and overlay-enhanced fisheye sky images. Overlays are generated through unsupervised, physics-informed cloud segmentation to highlight dynamic sky features. Empirical validation is performed using data from the Chilbolton Observatory, chosen for its temperate climate and frequent cloud variability. To simulate data-scarce conditions, models are trained on a single month (e.g., January) and evaluated across a temporally disjoint, full-year test set. Under this setup, the sequential and dual-parallel frameworks achieve RMSE values within 2-3 W/m^2 and 1-6 W/m^2, respectively, of a state-of-the-art ViT trained on the complete dataset. By combining physics-informed modelling with unsupervised segmentation, the proposed method provides a scalable and cost-effective solution for DHI estimation, advancing solar resource assessment in data-constrained environments.

Journal ref
Energy and AI 21 (2025), 100560
DOI
10.1016/j.egyai.2025.100560

Closest papers

Nearest neighbours by meaning across the whole index.

Bidirectional Multimodal Fusion of Sky Images and Time-Series for Solar Forecasting with Large Language Models

cs.LG ·2026-09-10 ·Ken Chen, Maneesha Perera, Wei Wang, Sachith Seneviratne, Hansani Weeratunge +1 more
similarity 0.75
Abstract

Short-term photovoltaic (PV) power and global horizontal irradiance (GHI) forecasts are essential for effective dispatch, reserve scheduling, and grid operations. At these forecasting horizons, errors are predominantly driven by cloud induced ramps: relying solely on historical numerical data may struggle to anticipate an incoming cloud, making ground-based sky images a crucial complementary physical signal. Furthermore, forecast performance is highly sensitive to location and local observing conditions, creating a strong need for site-specific data that are often scarce. Recently, large language models (LLMs) have demonstrated competitive performance and high data efficiency in time-series forecasting. Despite their success, existing LLM-based forecasting methods remain predominantly unimodal, relying primarily on historical numerical time-series data. Effectively incorporating sky imagery into an LLM-based forecasting framework remains under-explored and an open challenge. In this paper, we propose SolCloudLLM, an LLM-based multimodal forecasting framework. SolCloudLLM aligns sky-image patches with time-series patches and fuses their corresponding representations through bidirectional multimodal fusion, yielding a unified representation that is subsequently mapped into the embedding space of an LLM. Extensive experiments on the SIRTA and SKIPP'D datasets demonstrate that SolCloudLLM consistently outperforms the best baseline methods in MSE across all forecasting horizons, achieving a maximum relative MSE reduction of 25.4%. Stratified analysis further indicates that the benefits of multimodal fusion are concentrated primarily under cloudy conditions. Notably, SolCloudLLM achieves the best performance in nearly all few-shot settings, whereas other deep learning baselines experience substantial performance degradation and are frequently outperformed by the non-learning physical method.

0.07

GateFormer: Forecast-Space Decomposition for Intra-Hour Solar Irradiance Prediction from Sky Images

eess.IV cs.AIcs.CVcs.LG ·2026-10-22 ·David Hamlyn, Sunny Chaudhary, Jonathon Hare, Tasmiat Rahman
publishedbenchmark similarity 0.73
Abstract

Sub-hourly dispatch, balancing, and storage control increasingly depend on forecasts that anticipate rapid variability in solar generation. Accurate sky-image-based irradiance forecasting requires a combination of visual cloud dynamics with recent irradiance state while adapting to different short-term irradiance-change regimes. To address this, we introduce GateFormer, a GHI-conditioned dual-decoder ViT for intra-hour global horizontal irradiance (GHI) forecasting through forecast-space decomposition. GateFormer embeds observed GHI into the visual token stream, allowing interaction with sky-image patch features through self-attention. From this shared representation, a Trajectory Head predicts future GHI sequence, while a Delta Head predicts final-horizon change from the latest observed GHI. The two final-horizon forecasts are adaptively weighted in prediction space, with magnitude- and direction-supervision targeting the remaining error. On the NREL-TSI benchmark, GateFormer attains 125.01 W/m^2 RMSE at +30 min, improving over a state-of-the-art transformer baseline by 9.6% under paired-timestamp evaluation. Across NREL-TSI, Chilbolton, and SIRTA, GateFormer achieves the lowest final-horizon RMSE among the evaluated baselines, with forecast skill of 22.97-28.86% relative to smart persistence. Component ablations show adaptive weighting outperforms both individual decoders and fixed averaging, distinguishing GateFormer from conventional forecast combination, while the supervised correction further reduces RMSE by 1.68 W/m^2 and MAE by 4.86 W/m^2. Ramp-regime diagnostics show larger gains during operationally important positive ramps, reducing RMSE by 26.27 W/m^2. These results identify forecast-space decomposition as an alternative forecasting formulation, with potential relevance to other multimodal short-horizon forecasting tasks and deployment using only sky images and measured GHI.

0.26

SolarBench: A global solar energy nowcasting benchmark

cs.CV cs.LG ·2026-09-05 ·Yuhao Nie, Stephen Campbell, Quentin Paletta, Liwenbo Zhang, Tao Jing +16 more
benchmark similarity 0.70
Abstract

As the share of solar power grows, nowcasting weather-driven solar variability becomes critical for reliable energy system operation. State-of-the-art approaches increasingly apply deep learning to sky camera and geostationary satellite observations, but fragmented datasets and inconsistent evaluation make it difficult to determine whether reported improvements generalize across climates, cloud regimes, and photovoltaic (PV) systems. Here we introduce SolarBench, an open global benchmark for image-based solar nowcasting. SolarBench harmonizes more than six million sky and satellite images from 11 diverse sites spanning a decade, together with irradiance or PV output and auxiliary atmospheric data. An accompanying toolbox supports reproducible data access, processing, model development, and evaluation. Using SolarBench, we benchmark representative models and reveal a gap between average forecasting accuracy and the ability to capture rapid solar fluctuations. We further quantify predictability across cloud regimes and demonstrate data-efficient adaptation to new PV systems. SolarBench provides an extensible foundation for fair comparison and methodological innovation in solar nowcasting.

0.17

PBLH Estimation from Satellite Radiances via a Dual-Encoder Transformer

cs.CV cs.LG ·2026-09-23 ·Lorenzo Innocenti, Luca Catalano, Edoardo Arnaudo, Claudio Rossi, Salvatore Larosa +2 more
benchmarkcode/data similarity 0.66
Abstract · 13 pages, 3 figures, 2 tables. Extended version of the paper accepted at the MACLEAN workshop, ECML PKDD 2026. Code: https://github.com/link…

Estimating the Planetary Boundary Layer Height (PBLH) from satellite observations is a challenging regression problem due to the indirect relationship between top-of-atmosphere radiances and near-surface atmospheric structure. Progress has been limited both by the lack of architectures capable of handling the multimodal, spatially incomplete nature of satellite overpasses, and by the scarcity of suitable datasets. In this paper, we build upon the large-scale dataset pairing MetOp radiances with ERA5 PBLH labels that we introduced in our previous work, making three contributions. First, we establish a benchmark across eight approaches spanning pixel-wise regression, swath-wise sequence models, and convolutional and Transformer models operating on the full orbital passage. Second, we quantify what the resulting model actually relies on, using grouped Shapley decomposition over the input blocks. Third, we present the best-performing architecture found: a dual-encoder Transformer whose masked-input handling lets it operate in all weather conditions. The proposed model achieves MAE = 155.8 m on the held-out global test set, outperforming all baselines on every evaluation subset. On 30 out-of-distribution granules acquired on two days overlapping the TEAMx observational campaign, it achieves MAE = 165.3 m, outperforming a pixel-wise baseline trained on the same data (MAE = 197 m).

0.44

An Empirical Study on Learning Rooftop Solar Potential from Satellite Imagery Directly Without Explicit Geometry

cs.CV ·2026-10-30 ·Joshua Lee
code/data similarity 0.65
Abstract

Rooftop solar potential estimation often relies on LiDAR, digital surface models (DSMs), or pipelines that explicitly reconstruct roof geometry before estimating solar potential. We propose Direct Rooftop Solar Prediction (DRSP), an end-to-end formulation that maps satellite imagery and coarse regional sunlight information directly to application-level solar outputs. DRSP neither requires explicit geometric representations as input nor predicts roof orientation, slope, height, DSMs, or other geometric quantities as intermediate outputs, and therefore requires no geometric intermediate supervision. To enable large-scale study of this formulation, we develop a programmatic data-curation pipeline that converts per-panel information from the Google Solar API into usable-rooftop masks and spatial solar-energy targets, yielding two geographically distinct datasets containing approximately 60K rooftops from the U.S. Midwest and 87K from Texas. We demonstrate the feasibility of DRSP and systematically characterize the effects of geographic training distribution, model depth, image resolution, regional sunlight conditioning, and heatmap regression objectives. Overall, our results suggest that useful rooftop-scale solar information can be learned directly from 2D imagery without explicit geometric intermediate representations, offering a simpler path toward large-scale solar assessment. Our code and dataset are available at https://github.com/joshualee124/direct_rooftop_solar_prediction.

0.38

Edge-Adaptive Hiera-Seg Model for Building-Integrated Photovoltaic Power Forecasting

cs.CV cs.LG ·2026-11-15 ·Na Yu
similarity 0.64
Abstract · 13 pages, 7 figures, 1 table

Building integrated photovoltaic power forecasting is challenging because facade orientation, rooftop geometry, nearby buildings, vegetation, cloud movement, module temperature, inverter status and edge device reliability jointly affect power generation. This paper proposes an edge-adaptive HieraSeg model for short term BIPV power forecasting with multi-orientation visual-irradiance fusion. RGB facade images, optional thermal cues, irradiance measurements, module temperature, facade orientation, rooftop masks, shading labels, sky condition, historical power and edge device state are aligned into synchronous forecasting windows. HieraSeg extracts hierarchical regions from facade, rooftop, sky, shadow and background images, while the edge-adaptive module changes feature weights under missing frames, latency shifts and local shading changes. The segmentation branch identifies occluded module areas and separates facade, rooftop, sky and background evidence before the forecasting head estimates future BIPV output. Experiments are designed on a simulated multibuilding BIPV dataset with unseen building and unseen orientation tests. The model reports predicted power, confidence interval, dominant evidence, occlusion contribution, orientation contribution and energy-management review priority. This output is intended to show whether the forecast is driven by irradiance change, visible shading, orientation response, stale edge information or recent power history. The framework supports local forecasting at edge controllers with limited communication delay and computing resources, linking visual occlusion, irradiance dynamics and power history to traceable forecasting results for building energy management, microgrid scheduling and BIPV operation planning.

0.24

Horizon-specific Expert Fusion for Photovoltaic Power Forecasting

cs.AI ·2026-09-14 ·Xu Yuqing, Zhou Liguo, Sun Ze, Yu Lei, Jiang Mingming
similarity 0.61
Abstract

Short-term photovoltaic power forecasting requires models to represent regular solar cycles and weather-driven fluctuations whose importance changes with the forecast horizon. This study develops a hierarchical ensemble that combines temporal neural models, historical analogs, state climatology, and gradient-boosted trees. Solar geometry and numerical weather forecasts describe the expected generation conditions, while horizon-specific convex weights combine complementary predictions. A separate calibration step uses available historical forecast errors to account for recent bias. The framework is evaluated on public PVDAQ data at 15--240-minute horizons and on three GEFCom2014 solar zones at hourly horizons up to four hours. On PVDAQ, the ensemble achieves a daylight capacity-normalized mean absolute error of 4.315%, reducing error by 4.11% relative to full-feature LightGBM and by 6.03% relative to fine-tuned Chronos-2 under identical calibration. Expert-removal experiments identify redundancy within the ensemble. Across three training seeds on GEFCom2014, learned fusion improves upon equal weighting but performs comparably to LightGBM. The results support horizon-specific combination as a useful forecasting strategy while showing that its advantage over strong individual models depends on the dataset and evaluation period.

0.12

Tropospheric temperature and humidity profile retrieval from Meteosat Flexible Combined Imager based on deep learning

cs.LG physics.ao-ph ·2026-08-26 ·Alejandro Salgueiro, Johannes Rausch, Julie Thérèse Villinger, Angela Meyer
similarity 0.61
Abstract · 27 pages, 11 figures. Supporting information has 12 pages and 17 figures. Submitted for publication in Journal of Geophysical Research: Atmo…

The Meteosat Third Generation (MTG) Flexible Combined Imager (FCI) offers new opportunities for tropospheric temperature and humidity profiling, at higher spatio-temporal resolutions and expanded spectral coverage relative to its predecessor. Vertically resolved retrievals from broadband imagers are inherently challenging, and operational retrieval algorithms typically rely on numerical weather prediction (NWP) background fields to compensate for limited infrared spectral resolution, reducing the retrievals' independence. We develop a spatially aware deep learning framework to retrieve all-sky tropospheric temperature and humidity profiles from FCI, without forecast profiles as input. A Residual U-Net that exploits spatial context across all 16 FCI channels was trained on 14 months of collocated FCI observations and CERRA reanalysis targets over Europe. Validated against independent radiosondes, retrieved temperatures show biases below 0.4 K and standard deviations of 1.5-1.9 K. Retrieved relative humidity standard deviations range from 12-20 %, compared to 9-19 % for CERRA. Performance degrades modestly under clouds, with standard deviation increases below 0.4 K and 3 % RH beneath cloud tops despite limited direct radiative information. Ablation experiments show that spatial context improves retrievals, with the largest gains below cloud tops. Feature sensitivity analysis indicates broad consistency with FCI bands' established radiative transfer characteristics. Visible and near-infrared channels contribute despite not being commonly used in physics-based profile inversions. These results demonstrate that spatially aware deep learning models can extract statistically reliable tropospheric profiles from geostationary imager observations, independent of NWP forecast fields, enabling more rapid autonomous monitoring of the atmosphere.

0.18