CS Paper Radar

Characterizing Experiments with Synthetic MSI/HSI Data: A Structured Taxonomy for Agri-Food Research

Jasmine Battestin Nunes, Paula Dornhofer Paro Costa

eess.SP ·submitted 2026-09-22 12:42 UTC · v1

Abstract

Synthetic multispectral and hyperspectral data are increasingly used to address limited measurements, labels, and acquisition resources in spectral-imaging research. Yet the scope of the evidence produced by these experiments depends not only on the synthesis method, but also on sensing conditions, sample variability, data provenance, evaluation design, and downstream use. We present a literature-informed taxonomy for experiments using synthetic MSI/HSI data through spectral reconstruction and/or data augmentation, with a deliberate focus on agri-food research. The taxonomy is paired with a traversal procedure that connects available evidence, sensing constraints, experimental choices, and evaluation design to the forms of generalization that were actually tested, producing a compact experiment characterization record. Although primarily intended for studies with an existing research question, the same structure can also support earlier-stage planning when only an object of study, dataset, or sensing setup is initially defined. The framework was developed through question-driven reading, practical research experience, and iterative refinement against additional recent studies, and is offered as a basis for further use and refinement.

Comment
21 pages

Closest papers

Nearest neighbours by meaning across the whole index.

Can Generative AI Automate Data Extraction for Meta-Analysis? A Case Study on Intercropping Research

cs.AI ·2026-09-28 ·Zehao Lu, Xingguo Xiong, Wopke van der Werf, Thijs L. van der Plas, Ioannis N. Athanasiadis
published similarity 0.61
Abstract

Meta-analysis is the synthesis of information from multiple sources to arrive at an overarching conclusion. There is a large need for meta-analysis in agricultural research to synthesize what is known and analyze overarching patterns. Extracting data from published literature is, however, labor-intensive, time-consuming, and tedious, and is impeded by a lack of standardization in research design, units of measurement, and terminology. These challenges are particularly evident in the domain of crop species mixtures, also called intercropping. With the growing capabilities of LLMs, many recent attempts have focused on building systems and tools to automate data collection, yet rigorous assessment against human-labeled ground truth is often missing. In this research, we evaluate three LLM-based approaches---direct zero-shot prompting, a staged workflow, and a multi-agent system---with six open-weight models to extract data from the intercropping literature. The results are evaluated against the manually curated ground truth and through a downstream statistical analysis. Overall, direct zero-shot prompting is the strongest and most consistent approach, achieving the highest mean similarity-adjusted F1 of 0.577, although none of the approaches is close to fully accurate. In the downstream analysis, most model--approach combinations recover the direction of the relationship between the predictor and outcome variables, but do not estimate its magnitude accurately.

0.55

Synthesizing hyperspectral Data using generative Models to train spectral Unmixing Methods for low-cost Crop Residue Cover Mapping

cs.LG cs.CVeess.SP ·2026-11-18 ·Ege Artan, Shuo Chen, Andrew Peng, Sammuel Aldrich Karya, Kyaw Thiha
published similarity 0.59
Abstract · 8 pages, 3 figures. Published in the XXV ISPRS Congress 2026

The spectral range of the Field Imaging Nanosatellite for Crop residue Hyperspectral Mapping (FINCH), developed by the University of Toronto Aerospace Team's Space System Division, poses significant challenges for hyperspectral unmixing to determine crop residue cover fractional abundances. The severe lack of standard indices necessitates the use of complex, data-driven unmixing models. Complex unmixing models require dense, well-distributed manifolds, whereas ground-truth datasets are sparse and expensive. To better leverage the information content in existing ground-truth datasets, this study presents a framework that decouples interpolation and abundance-mapping estimation via an intermediary conditional data generator, contrasting with traditional single-model unmixing pipelines. This decoupling enables the inclusion of physical prior knowledge about spectra, which is not natively accessible to unmixing models, thereby artificially expanding the dataset to serve as a Monte Carlo approximation of the population risk. To test this hypothesis, we have proposed two generator models: a 1D Conditional U-Net with Conformer Layers (GD-Streamline) and a Dual-Path Transformer Conditional Variational AutoEncoder (TCVAE), and two unmixing models: Multi-Layered Perceptrons (MLP) and Fourier Neural Operators (FNO). The results of the study indicate a need for rigorous integration and introduction of spectral priors within the scope of the proposed decoupling framework to prevent domain and mode collapse and hallucination by the generator models; otherwise, this leads to falsely approximated data manifolds, ultimately resulting in unsatisfactory and out-of-distribution unmixing performance.

0.35

Earth Observation Foundation Models for Terrestrial Ecohydrology: From Representation Learning to Process Inference

cs.LG cs.CVphysics.bio-ph ·2026-08-15 ·Yi Yu, Jian Peng, Yucheng Lin, Trevor F. Keenan, Thomas F. A. Bishop
benchmark similarity 0.55
Abstract

Earth observation foundation models (EOFMs) are emerging as reusable representation frameworks for data-driven retrieval, prediction and process modelling within ecohydrology, which integrate EO, meteorological forcing and process models to characterise coupled water, energy and carbon dynamics in vegetation and soil across scales. However, there is yet to be an ecohydrology-specific synthesis assessing the EOFM relevance, application evidence or evaluation requirements under uncertain reference data, scale mismatch and temporal dependence. Here, we develop a framework for determining when EOFMs support interpretable inference and identify a mismatch between EOFMs and ecohydrological requirements. Firstly, an observation-to-inference hierarchy shows that relevance depends on target-specific sensing pathways, spatial-temporal support and traceable uncertainty. Secondly, a meta-analysis shows that pretraining is dominated by reflected optical and active-microwave data, with sparse thermal coverage and no passive-microwave-emission sources. Thirdly, our synthesis of ecohydrological applications finds strongest support for spatial context, label-efficient adaptation and hybrid workflows. Evidence declines with inference depth; independent validation of fluxes, coupled dynamics, event trajectories, calibrated uncertainty and decision benefits remains sparse. Fourthly, our benchmark audit finds stronger coverage of fair adaptation and reproducibility in general EOFM suites, and of process targets, direct reference evidence and distribution shifts in ecohydrological evaluations; physical consistency and uncertainty remain weakly assessed. These findings motivate a process-aware framework aligning EOFM design and evaluation with the target variable, observation pathway and process timescale, supporting trustworthy monitoring and interpretation of coupled water, energy and carbon dynamics.

0.04

ADA 2025: A Multimodal Dataset for Water-Deficit Stress Detection in Sugar Beets

cs.CV ·2026-10-16 ·Piper Powell, Robert Franz, Amos Smith, Isaak Ihorst, Joachim Hertzberg +1 more
code/data similarity 0.54
Abstract · 6 pages, 4 figures, 1 table. Accepted at the 2026 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor). Data…

Research and development into artificial intelligence (AI) and machine learning (ML) applications in crop management are hindered by a lack of suitable public datasets. Whereas applications in disease and pest detection benefit from a comparatively higher amount of available data, applications like water-deficit stress detection often have very few if any public datasets available. Moreover, most datasets that are available are single-modality and frequently RGB-only, leaving the early detection and enhanced performance offered by other methods, and particularly by multimodal approaches, out of reach. Datasets with accurate metadata, higher numbers of plants, and smaller intervals between data points are also scarce. To address this, we publicly release ADA 2025, a >500 GB comprehensive multimodal dataset of young sugar beet plants under healthy and water-deficit conditions combining thermal, hyperspectral, RGB, and depth imagery along with relevant environmental and metadata and both relative water content (RWC) and leaf thickness as ground truth sources. Exemplary results showcase the dataset's validity in terms of differences detectable between stressed and healthy plants.

0.36

Minerals in the Wild: A Hyperspectral-XRF Dataset for Elemental Composition Estimation

cs.CV cs.LG ·2026-08-31 ·Eleftheria Tetoula-Tsonga, George Arvanitakis, Theodoros Giannakas
dataset similarity 0.54
Abstract

Rapid mineral characterization is essential for applications ranging from mineral exploration to industrial ore processing. To this end, Hyperspectral Imaging (HSI) has emerged as a promising sensing modality thanks to its fine spectral resolution, enabling mineral discrimination in both close-range and remote sensing settings. However, the scarcity of publicly available datasets with reliable ground-truth labels hinders the development and evaluation of HSI-based mineral identification methods. We release Minerals in the Wild, a multi-purpose dataset comprising 1,132 rock specimens collected across Europe. For each specimen, we provide an HSI acquisition together with an elemental characterization obtained via an XRF sensor. We define the task of elemental characterization on our dataset and propose a pruning mechanism that removes distant signatures from the USGS dictionary prior to a convex optimization approach for matching HSI pixels with USGS spectral signatures. Finally, we empirically show that our approach outperforms simpler baselines.

0.56

SpecUltra: A Unified Foundation Model for Heterogeneous Spectral Images

cs.CV ·2026-11-01 ·Weili Kong, Baisen Liu, Xiaojun Bi, Jiaming Pei
similarity 0.53
Abstract

Spectral imagery, including hyperspectral image (HSI) and multispectral image (MSI), exhibits substantial heterogeneity in band numbers, wavelengths, spectral resolutions, hindering unified modeling and cross-sensor transfer. From the perspective of imaging physics, these heterogeneous observations can be regarded as discrete measurements of an underlying continuous spectrum acquired through different sampling processes. Motivated by this perspective, we propose SpecUltra, a unified foundation model for heterogeneous spectral images. Specifically, SpecUltra employs 3D convolutional tokenization and wavelength-aware 3D RoPE to encode relative wavelength and spatial relationships, enabling fine-grained spatial-spectral modeling within a shared coordinate system. We further construct MultiHyper-200K, a large-scale multi-sensor HSI dataset, and pretrain SpecUltra using auxiliary-token-assisted mask image modeling (MIM), where a level token indicates the radiometric product type and register tokens absorb sensor- and acquisition-dependent variations. To bridge the spectral-granularity gap between HSI and MSI, we introduce Spectral Kernel Transformation (SKT), which transfers pretrained knowledge in parameter space by constructing MSI embedding kernels from fine-grained HSI-pretrained kernel slices. This design preserves pretrained spectral priors while maintaining compatibility with the pretrained backbone. Experiments on HSI and MSI classification and segmentation benchmarks demonstrate state-of-the-art or competitive performance across disparate sensors and downstream tasks.

0.24

Foundation Models Meet Agriculture: Challenges Beyond Pretraining

cs.LG ·2026-08-31 ·Vishal Nedungadi, Xingguo Xiong, Marc Rußwurm, Ioannis N. Athanasiadis
similarity 0.53
Abstract

Global food security and sustainable climate action increasingly rely on robust, scalable agricultural monitoring. Earth observation foundation models have emerged as powerful, label-efficient tools across general remote sensing domains, yet early attempts to deploy them for agricultural applications have yielded surprisingly poor results. We hypothesize that this performance gap stems from the extreme heterogeneity of agricultural landscapes and the inherent inability of current earth observation foundation models to adapt to task-specific nuances. In this work, we systematically evaluate two critical bottlenecks hindering the deployment of foundation models in agricultural tasks, benchmarking two earth observation foundation models, a foundation model designed for tabular data, and conventional supervised baselines across seven real-world agricultural datasets spanning yield prediction, phenology estimation, and crop classification. First, we identify a pretraining-deployment modality gap: agricultural downstream tasks frequently require diverse, non-imagery data modalities that earth observation foundation models are architecturally unequipped to ingest, while a foundation model built for tabular data handles this heterogeneity more naturally. Second, we formalize the agricultural task space across five structural axes to demonstrate why current models fail to generalize reliably, resulting in highly unstable model rankings across evaluation settings. By characterizing these structural and modal gaps, our insights highlight the friction between general-purpose architectures and specialized agricultural downstream data, providing a strategic roadmap for developing the next generation of domain-aware foundation models.

0.05

UnMixMamba: Unmixing Guided State Space Subpixel Mapping of Crops from MODIS Time Series

cs.CV ·2026-10-28 ·Motasem Alkayid, Yimin Zhu, Naser El-Sheimy, Lincoln Linlin Xu
similarity 0.53
Abstract

Subpixel mapping (SPM) of crop types from \SI{250}{m} Moderate Resolution Imaging Spectroradiometer (MODIS) time series to field-scale Agriculture and Agri-Food Canada (AAFC) labels supports acreage estimation and agricultural monitoring. Three difficulties remain: (i)~spectral-temporal coupling and mixed pixels that break direct coarse-to-fine classifiers; (ii)~intra-class endmember variability that a single spectrum per crop cannot capture; and (iii)~uniform refinement on the high-resolution (HR) grid, which treats every subpixel the same, so most of the effort goes to easy pixels while errors remain in the minority class. UnMixMamba addresses them as follows. For~(i), a temporal-spectral-spatial decoupled (TSSD) factorization estimates simplex fractions and drives AAFC classification from the same mixture field, instead of mapping low-resolution (LR) observations directly to HR logits. For~(ii), \(\times 8\) abundance super-resolution and a learnable per-crop variant endmember bank reconstruct the MODIS cube at LR and HR under a linear mixing model. For~(iii), the predicted crop fractions decide where a fixed Mamba token budget is spent, so HR refinement focuses on the minority class instead of the full grid. On Alberta 2015 under a fixed spatial-block protocol, UnMixMamba reaches OA 0.8397, mIoU 0.6640, AA 0.7837, and Kappa 0.7715, ahead of ST-DRes, ESPCN, FineCrop, AgriFM, and classical SPM baselines. The task is closed-set four-crop mapping inside a known agricultural mask.

0.30