CS Paper Radar

Morpho-Spectral Projection Analysis (MSPA) for Plant Spectroscopy

Lucas Prado Osco, Marcelo Santos da Silva, Filipe Vieira Silva, Liang Zhao, Ana Paula Marques Ramos, Ednaldo Jose Ferreira, Roberto Dantas de Medeiros, Amaury Burlamaqui Bendahan, Lucio Andre de Castro Jorge

cs.LG ·submitted 2026-11-18 15:25 UTC · v1
code/dataunusual

Abstract

Plant reflectance spectroscopy across the visible, near-infrared, and shortwave infrared domain encodes foliar biochemical and anatomical traits into contiguous spectral signatures with locally differentiable structure; however, extreme inter-band collinearity and unfavorable sample-to-feature ratios challenge predictive modeling. Conventional dimensionality reduction methods routinely adopted in spectroscopy, are column-permutation invariant: they treat adjacent wavelengths as unordered tabular columns and discard the sequential differential geometry of absorption features. Here, we propose Morpho-Spectral Projection Analysis (MSPA), a supervised dimensionality reduction operator that projects contiguous spectra into a compact morpho-spectral subspace through three complementary blocks: (i) additive continuum-removed absorption depth, area, and red-edge inflection kinematics; (ii) local multi-order singular eigenmodes (0th, 1st, and 2nd derivatives) partitioned across contiguous radiative sub-windows; and (iii) global Fisher-weighted canonical subspaces regularized via trace-normalized isotropic covariance shrinkage. We validated MSPA across 8,963 spectra spanning five plant systems, coupled with six classical machine learning classifiers and four Tabular Foundation Models (TabPFN v3, TabICL v2, TabDPT v1.2, and TabFM). Despite compressing the 1,580-band spectrum by 96%, MSPA outperformed the full spectrum in 92.0% of all benchmarks and conventional reducers in 90.0% of all model-crop evaluations, raising the global mean macro-F1 from 0.7756 (full spectrum) to 0.8520, accelerating in-context inference by up to 126.5x. Designed as a self-contained spectral projection operator, the MSPA method can be directly integrated into spectroscopy workflows via the open-source code available at GitHub: https://github.com/Lucas-Osco/mspa.

Closest papers

Nearest neighbours by meaning across the whole index.

Turning spectra into images improves plant trait retrieval with 2D-CNNs

cs.CV cs.LG ·2026-08-17 ·Javier Lopatin, Teja Kattenborn, Eya Cherif, Sebastián Moreno
similarity 0.60
Abstract · 38 pages, 14 figures, 11 tables. Supplementary material appended after the references. v2: adds a per-image vs global scaling ablation, expa…

Hyperspectral reflectance spectroscopy enables non-destructive estimation of plant functional traits, yet current deep learning approaches process spectra as one-dimensional sequences, which limits how they capture long-range inter-band dependencies. We asked whether transforming 1D spectra into 2D image representations improves multi-trait prediction with convolutional neural networks (CNN). We compared nine transformations using EfficientNet-B0 on the GreenHyperSpectra dataset (7,897 labeled spectra, eight traits, 400-2450 nm), benchmarked against published 1D CNN results on the same split. Trained from scratch, the simplest transformation, a direct Reshape of the spectrum into a 2D grid, performed best ($R^2 = 0.684 \pm 0.001$) and improved on the state-of-the-art 1D baseline ($R^2 = 0.587$, $+0.097$). We then pretrained a 2D masked autoencoder (MAE-2D) on 139,000 unlabeled spectral images. Linear probing, which freezes the encoder and trains only a multilayer perceptron head, reached $R^2 = 0.646$ and exceeded every 1D self-supervised counterpart, including the fine-tuned MAE-1D ($R^2 = 0.641$). Under cross-dataset evaluation all models lost most of their accuracy and none beat the 1D baseline significantly. To identify which wavelengths drive each prediction, we applied Integrated Gradients and Grad-CAM and unfolded band importance back to the spectral axis. Protein ($r = 0.45$) and leaf water ($r = 0.33$) agreed with sensitivities simulated by the PROSAIL radiative-transfer model, while carotenoids ($r = 0.06$) and leaf area index ($r = -0.11$) did not, showing that the model reads established leaf chemistry for traits with sharp absorption features. The representational advantage of 2D spectral images, rather than architectural complexity or ImageNet pretraining, drives the gain over 1D approaches.

0.24

SpecUltra: A Unified Foundation Model for Heterogeneous Spectral Images

cs.CV ·2026-11-01 ·Weili Kong, Baisen Liu, Xiaojun Bi, Jiaming Pei
similarity 0.57
Abstract

Spectral imagery, including hyperspectral image (HSI) and multispectral image (MSI), exhibits substantial heterogeneity in band numbers, wavelengths, spectral resolutions, hindering unified modeling and cross-sensor transfer. From the perspective of imaging physics, these heterogeneous observations can be regarded as discrete measurements of an underlying continuous spectrum acquired through different sampling processes. Motivated by this perspective, we propose SpecUltra, a unified foundation model for heterogeneous spectral images. Specifically, SpecUltra employs 3D convolutional tokenization and wavelength-aware 3D RoPE to encode relative wavelength and spatial relationships, enabling fine-grained spatial-spectral modeling within a shared coordinate system. We further construct MultiHyper-200K, a large-scale multi-sensor HSI dataset, and pretrain SpecUltra using auxiliary-token-assisted mask image modeling (MIM), where a level token indicates the radiometric product type and register tokens absorb sensor- and acquisition-dependent variations. To bridge the spectral-granularity gap between HSI and MSI, we introduce Spectral Kernel Transformation (SKT), which transfers pretrained knowledge in parameter space by constructing MSI embedding kernels from fine-grained HSI-pretrained kernel slices. This design preserves pretrained spectral priors while maintaining compatibility with the pretrained backbone. Experiments on HSI and MSI classification and segmentation benchmarks demonstrate state-of-the-art or competitive performance across disparate sensors and downstream tasks.

0.24

Cross-Spectral Dense Correspondence for Multimodal Spectral Medical Imaging

cs.CV cs.AI ·2026-08-28 ·Eric L. Wisotzky, Jost Triller, Simon W. Härtl, Oliver T. Bruns, Peter Eisert +1 more
publishedbenchmark similarity 0.54
Abstract · Accepted at 2nd Data Curation & Augmentation in Medical Imaging Workshop at ECCV 2026

Precise dense correspondence is a fundamental prerequisite for multimodal spectral imaging systems that fuse disparate wavelength ranges for subsequent analysis in medical and scientific imaging. Corresponding image points are often observed with non-overlapping spectral sensitivities, leading to wavelength-dependent contrast changes, intensity inversions, and appearance shifts for which dense ground truth is difficult to obtain and conventional RGB-based training data provides only limited supervision. We address this data gap by introducing a sensor-agnostic cross-spectral modulation protocol on established correspondence benchmarks with intensity input projection, and by proposing a synthetic cross-spectral correspondence benchmark simulating physically plausible radiometric differences. Evaluation on several modern dense correspondence backbones trained with our unified cross-spectral protocol showed substantial improvements under severe spectral mismatch while maintaining performance on standard RGB benchmarks. Ablation experiments show that view-dependent channel selection and nonlinear radiometric transformations provide complementary robustness, indicating that the primary limitation of existing models is not their structural matching capacity but the mismatch between training distribution and spectral characteristics of the target image pair. Qualitative evaluations on heterogeneous medical spectral acquisition systems demonstrate the practical relevance of the proposed training data augmentation protocol as an enabler for spatially coherent spectral fusion in HSI workflows.

0.34

A multi-level preprocessing and modelling framework for spectral imaging of microplastics

cs.CE stat.CO ·2026-08-18 ·Zina-Sabrina Duma, Tenzin Tsering, Sara Heikkinen, Tuomo Soininen, Tuomas Sihvonen +2 more
publishedunusual similarity 0.52
Abstract

Spectral imaging provides chemically specific and spatially resolved analysis of microplastics, but its routine application is hindered by large data volumes, acquisition artefacts, spectral variability, and misidentification of polymers due to alike spectra. This study proposes a multi-level preprocessing and modelling framework for FT-IR spectral imaging of microplastics that integrates image-level, tile-level, and spectral-level corrections with scalable identification strategies. Image-level variation associated with changing acquisition conditions was done with latent variable selection, while a background-based tile correction reduced illumination-related artefacts. Spectral preprocessing combined baseline correction, smoothing, derivative calculation, normalization, and wavelength selection, and only particle spectra were retained for further analysis to improve computational efficiency. For scalable identification, clustering was applied to particle spectra and spectral library matching was performed on cluster centroids instead of individual pixels. Among twelve evaluated matching strategies, a sign-invariant derivative-based cosine similarity method achieved perfect classification accuracy for polystyrene (PS), polyethylene terephthalate (PET), polyethylene (PE), and polypropylene (PP). The clustering-based workflow also produced more spatially coherent particle maps than direct software-based matching while substantially reducing processing time. The framework was evaluated for supervised classification-based MP indentification. These results show that multi-level correction combined with cluster-centroid spectral matching improves the robustness, efficiency, and interpretability of spectral-imaging-based microplastic identification.

0.46

S3D-Mosaic: Subgradient-Guided Cross-Band Structure Transfer for Spectral Demosaicing

cs.CV ·2026-10-17 ·Weiqiang Zhao, Xian Li, Yuzhe Gui, Tianzhu Liu, Yanfeng Gu
code/data similarity 0.52
Abstract

Multispectral filter array (MSFA) imaging enables snapshot spectral acquisition, but its per-band sparse sampling causes severe spatial aliasing. Although missing spatial structures can be inferred across bands, wavelength-dependent contrast variations make direct cross-band transfer unreliable. Existing model-based methods lack explicit magnitude-invariant mechanisms, and learning-based methods require layout-specific training or per-image tuning, limiting generalization. To address these issues, we propose S3D-Mosaic, a model-based spectral demosaicing method built upon a subgradient-guided cross-band structure transfer framework. We first establish a subgradient-guided regularization framework with a generalized cosine-similarity interpretation, providing a convex mechanism for magnitude-invariant information transfer. Then we design a compact Directional Sparse Equivariant Difference Convolution (DSE-DConv) for band-level extraction of orientation-selective, high-frequency structure features at low computational cost. Instantiating the framework with DSE-DConv on spectrally adjacent bands yields a cross-band self-structure similarity regularizer for spectral demosaicing, which we optimize via a dual-scan ADMM algorithm that decouples the bidirectional dependency while propagating magnitude-robust structural information across bands. With DSE-DConv's two parameters trained once via self-supervision, S3D-Mosaic outperforms state-of-the-art unsupervised methods on CAVE and NTIRE 2022, and generalizes robustly to varying MSFA layouts, pan-mosaic fusion, and real-world UAV data with fixed reconstruction hyperparameters. The source code is available at \url{https://github.com/bestwishes43/S3D-Mosaic}.

0.33

Robust small-molecule identification from incomplete, degraded, and inconsistent spectra using multimodal mixed-condition training

cs.LG ·2026-09-13 ·Bowen Gao, Lei Zhu, Yiying Wang, Wenjie Yu
similarity 0.52
Abstract · 17 pages, 7 figures, 2 tables. Supplementary information: 12 pages, 3 figures and 11 tables, provided as an ancillary PDF. Revised content

Reliable small-molecule identification often requires complementary evidence from multiple spectroscopic measurements. In practice, however, spectra may be unavailable, degraded by measurement-related variations, or even incorrectly associated with a sample, thereby hindering accurate molecular identification. Herein, we propose a multimodal mixed-condition training strategy that accommodates missing, degraded, and mismatched measurements for small-molecule structure identification. The strategy incorporates chemical and spectroscopic knowledge through predefined missing-input configurations, modality-specific spectral perturbations, and chemically informed spectrum replacements. Models were trained on 635,441 samples comprising mass spectrometry (MS), infrared (IR), and nuclear magnetic resonance (NMR) simulated spectra from the Multimodal Spectroscopic Dataset (MSSD). They were then systematically evaluated on 79,462 held-out samples across 30 views designed to represent variations in spectra. A controlled comparison of complete-input and mixed-condition training under concatenation and mixture-of-experts (MoE) fusion showed that the training strategy was the principal source of improvement. For MoE, mixed-condition training increased the mean reciprocal rank (MRR) by 6.08% (from 0.9203 to 0.9763) and the top-1 molecular identification rate by 7.67% (from 89.50% to 96.36%). Notably, under single-modality inputs, IR MRR increased 2.15-fold (from 0.4337 to 0.9307), while MS MRR increased 2.31-fold (from 0.3711 to 0.8575). With the proposed strategy, complete-input performance remained high, while sample-level mismatch detection also improved. Together, these results highlight the potential of multimodal mixed-condition training for practical molecular identification by explicitly addressing incomplete, degraded, and mismatched measurements encountered in real-world analysis.

0.40

MWIR-4-Plastic: The Identification of Complex End-of-Life Industrial Plastic using Mid-wave Infrared Hyperspectral Imaging and Machine Learning

cs.CV cs.LG ·2026-08-28 ·Elias Arbash, Andréa de Lima Ribeiro, Filipa Simões, Ahmed Jamal Afifi, Aldino Rizaldy +7 more
publishedbenchmarkdatasetcode/data similarity 0.52
Abstract · 15 pages

The automated sorting of shredded black plastics from end-of-life (EOF) industrial waste presents a significant challenge in recycling facilities, primarily due to the limitations of current sensing and analytical approaches. Existing studies predominantly rely on single-point contact-based mid-infrared spectroscopy or laboratory hyperspectral imaging (HSI) setups, which fail to provide the spatially resolved analysis necessary for fast, bulk processing. Moreover, available datasets are laboratory-controlled and focus on intact rather than shredded plastics, hindering further recycling refinement. Black industrial plastics, in particular, are underrepresented, while most classification pipelines depend on manual region selection and rule-based spectral matching, neglecting spatial information and modern deep learning (DL) methods. To address these gaps, we introduce the first publicly available HSI dataset of shredded black plastics from EOF vehicle, comprising four industrial polymers across 13 co-registered RGB, VNIR, SWIR, and MWIR scenes and their segmentation pipeline. We developed a multi-modal spectral-spatial framework that integrates foreground isolation, pixel-wise classification, and object-level majority voting. By adapting advanced hyperspectral transformers from earth observation and incorporating chemometric band selection, we achieve accurate classification of complex black plastics. The study establishes the first comprehensive benchmark using nine processing methods, including chemometric, machine learning, and DL architectures. To ensure reproducibility, the complete dataset and methodologies are publicly released, establishing a benchmark for a hyperspectral object-analysis pipeline in industrial inspection.

0.31

MM-Spectrum: Multimodal Multi-spectral Molecular Structural Elucidation with a Stable MoE Framework

cs.LG ·2026-08-27 ·Hai-tao Yu, Nan Min, Zheng Fang, Hongyu Zhan, Yusen Tan +2 more
top venue similarity 0.52
Abstract · Accepted to ICML 2026

Inferring molecular structures from multimodal spectroscopic measurements requires integrating complementary yet highly heterogeneous signals. However, the common paradigm of directly concatenating multispectral sequences can exhibit anomalous performance degradation, primarily due to pronounced heterogeneity and the resulting multimodal imbalance across modalities. As a remedy, we propose MM-Spectrum, a sparse Mixture-of-Experts framework tailored for multimodal multispectral spectra-to-structure elucidation. To better match the information characteristics under multispectral imbalance, MM-Spectrum introduces an explicit modality-aware routing mechanism that exposes spectral identity to the router in addition to token content representations. Moreover, it incorporates shared and interaction experts, together with heterogeneous expert capacities, to extract multispectral modality-unique and cross-modal synergistic information while suppressing noise-induced interference. Across full-modality, bimodal, and missing-modality settings on molecular structural elucidation, MM-Spectrum achieves consistent and substantial improvements, supported by ablation studies and interpretability analyses.

0.43