CS Paper Radar

Band-Selection Stability and Semantic Segmentation Performance: A Study on Hyperspectral City

Jiarong Li, Imad Ali Shah, Enda Ward, Martin Glavin, Edward Jones, Brian Deegan

cs.CV ·submitted 2026-09-25 10:13 UTC · v1
published

Abstract

Resource constraints make high-dimensional hyperspectral imaging challenging in autonomous perception, motivating the use of band selection methods. However, the sensitivity of band-selection methods to sampled data and their relationship to semantic segmentation models (SSMs) remain underexplored. This study evaluates six band selection methods on ten independently sampled, class-balanced region-of-interest (ROI) sets, yielding 60 top-25 band subsets from the Hyperspectral City V2 (128 bands: 450-950nm) dataset. Top-$K$ bands ($K\in\{3,5, ... 13\}$) from the first three ROI sets are evaluated with three SSMs against the corresponding 128-band baseline. Experiments show that intra-method stability is method-dependent: Sim-LP shows the highest stability (pairwise Jaccard similarity) and, together with JMIM+CSNR, yields the best segmentation results. Top-$K$ based SSMs remain competitive with baselines, with gains of up to 2.01 mIoU and 1.72 mF1 points, and 18-22x faster CPU inference for $K=9$. However, performance does not improve monotonically with $K$, and stability shows no consistent association with SSM performance. These findings suggest that intra-method stability is informative but an unreliable indicator of downstream segmentation performance, highlighting the need to evaluate band-selection methods across repeated samples, subset sizes, and SSMs.

Comment
Accepted for IEEE WHISPERS 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Spatial Uncertainty Analytics for Budget-Limited Inspection of Hyperspectral Scenes

cs.CV ·2026-10-31 ·Ritesh Sharma, Yuichi Motai
benchmark similarity 0.60
Abstract

Land-cover maps classified from wide-area hyperspectral surveys are least reliable along class boundaries and in ambiguous regions, so their errors are spatially structured. We study how this structure can direct a limited verification budget: when a low-altitude UAV or a ground crew can visit only a handful of sites, where it should go depends on how classifier uncertainty is arranged across the scene and how a fixed sensor footprint tiles that arrangement. We estimate a per-pixel uncertainty surface from the disagreement of an ensemble of spectral classifiers and select sites by greedy, field-of-view-aware coverage of the uncertainty mass. On seven benchmark scenes, the highest-uncertainty decile of pixels is 2.6 to 7.9 times richer in errors than the scene as a whole, and the coverage greedy covers up to 57% of the classifier's errors with 15 sites, 1.7 to 18 times as many as uncertainty-agnostic sampling of the same labelled region. A non-overlapping selection of the footprints richest in uncertainty does equally well, so the gain comes from scoring whole footprints by uncertainty mass. The advantage persists under spatially blocked evaluation and with a spectral-spatial classifier, although disagreement then ranks errors less well. It requires restricting the surface to labelled land cover: over the whole scene, disagreement peaks on unlabelled ground, where inspections cannot be scored. The best policy for covering connected high-uncertainty regions depends on the scene and on the threshold that defines them, and Gaussian-process selection covers less uncertainty than the coverage greedy on every scene. The pipeline runs on commodity CPUs.

0.32

From Proper Scoring Rules and Edge Biases to Mumford-Shah Models: Robust Unsupervised Hyperspectral Image Segmentation

cs.CV ·2026-10-20 ·Fabian Conrad, Benjamin Berkels
similarity 0.59
Abstract

Hyperspectral images provide a rich representation of the underlying spectrum for each pixel, allowing for a pixel-wise classification/segmentation into different classes. As the availability of labeled hyperspectral images is limited and the acquisition of labeled training data is time-consuming, unsupervised segmentation methods are important in hyperspectral image segmentation. Spectral variability, mixed pixels, noise as well as high correlation between neighboring spectral bands make this task challenging. We present a feature-agnostic, unsupervised hyperspectral segmentation framework based on the piecewise-constant Mumford-Shah functional. Our framework turns a strictly proper scoring rule and a family of parametric probability distributions into an indicator function for the Mumford-Shah functional, and we naturally generalize some previous Mumford-Shah based models for image segmentation this way. Our focus will be on heavy-tailed elliptical distributions, the negative log-likelihood score and the score based on the beta-divergence to deal with the heavy outliers in hyperspectral imaging. Using the interplay between the calculus of variations and statistics, we also propose a robustness step to estimate the statistical parameters of these heavy-tailed elliptical distributions. In addition, an edge bias, which contains prior boundary information, is incorporated into both the data fidelity term and the perimeter term of the Mumford-Shah functional. Numerical experiments show that, to the best of our knowledge, our method achieves the highest overall accuracy among fully unsupervised methods on three of the four standard benchmarks (Indian Pines, Kennedy Space Center, Pavia University and Salinas) evaluated on the full scenes. It also improves upon our previous Mumford-Shah based model.

0.53

Hierarchical Filter Band Selection for Multispectral Object Classification

eess.IV ·2026-09-20 ·Katja Kossira, Jürgen Seiler, André Kaup
published similarity 0.58
Abstract · This paper was publsihed at IEEE International Conference on Image Processing (ICIP) 2026, Tampere, Finnland

Multispectral camera arrays capture image data in various spectral bands, enabling image acquisition beyond human perception. These systems are widely used in medical, agricultural, environmental, and remote sensing applications. However, not all recorded bands are needed for classification tasks, thus reducing them can lower hardware complexity and cost. The conditional filter band selection algorithm addresses this by selecting low-noise, non-redundant bands to minimize the number of filters and cameras. This paper improves the approach by introducing a second classification stage that estimates object material in addition to the object label. This information is merged by a decision-tree based band selection strategy. The proposed method achieves a 28.9% relative reduction in classification error on the SMM50 dataset compared to the state-of-the-art. Moreover, for the same classification accuracy, the required number of cameras is reduced from 7 to 4, demonstrating that the proposed approach improves performance while significantly lowering hardware requirements.

0.31

Token Clustering and Semantic Sequence Mamba for Hyperspectral Image Classification

cs.CV ·2026-09-23 ·Yimin Zhu, Mahmood Elahi, Lincoln Linlin Xu
benchmark similarity 0.56
Abstract

Although hyperspectral images (HSIs) provide rich spectral-spatial information, accurate pixel-level classification remains challenging because of spectral-spatial heterogeneity and complex spatial structures. Existing vision state space models (Mamba) typically construct sequences according to predefined spatial neighborhoods, without explicitly accounting for semantic similarity or spatial non-stationarity. To address this limitation, we propose Token Clustering and Semantic Sequence Mamba (STMamba), which organizes sparse tokens into semantically coherent sequences for hyperspectral image classification with the following features. First, at the macro level, a hierarchical encoder decoder progressively selects semantic tokens with the Token Clustering Module (TCM) and restores dense features using a parameter-free Cross-scale Neighborhood Attention (CNA) Upsampler. Second, at the micro level, TCM first identifies representative cluster centers through density-aware clustering and estimates soft memberships based on feature similarity. A quadtree-based dynamic selection strategy then retains sparse and spatially distributed tokens from each semantic cluster, forming coherent semantic-token sequences while reducing redundant pixel-wise representations. Third, parallel Spatial and Spectral Semantic-wise Sequencing Mamba (SWSM) modules capture complementary long-range spatial and spectral dependencies within homogeneous semantic token sequences while suppressing irrelevant interactions across heterogeneous regions. Experimental results on three large-scale benchmark datasets demonstrate that STMamba outperforms the SOTA methods with respect to quantitative and qualitative results.

0.19

Ten Architectures, One Error: Shared Failure Modes in Hyperspectral Classification under Spatially Disjoint Evaluation

cs.CV cs.LG ·2026-09-01 ·Ehsan Faghih, Fatemeh Ashrafi, Marguerite Moore, Zahra Saki
benchmark similarity 0.55
Abstract

Hyperspectral image classification still relies heavily on random pixel splits within a single scene. The Salinas dataset, randomly split, is among the most widely used datasets for comparing different architectures. However, under a random split method, a large fraction of test pixels fall immediately adjacent to a training pixel, which inflates reported accuracy. This work introduces a leakage-free evaluation protocol linking spatial separation to the model's receptive field. Applying this protocol across ten different architectures, including classical, spectral, spectral-spatial, transformer, vision-backbone, and state-space families, shows that Macro-F1 drops by 0.147 on average and model rankings change by as many as five places. Furthermore, leakage-free evaluation limits which architectures can be tested on a given benchmark. Since each partition supports patches only within a finite radius, reporting this radius alongside the receptive field is essential for fair comparison. In addition, this study reveals that all ten architectures misclassify largely the same pixels, pointing to a spectral ambiguity in the data that none of them resolves.

0.22

Data Leakage in Patch-Based Hyperspectral Image Classification: Quantifying the Impact of Spatial Overlap

cs.CV ·2026-10-06 ·Mohammed Q. Alkhatib
publishedcode/data similarity 0.55
Abstract · paper accepted for presentation at IEEE-WHISPERS

Patch-based learning improves hyperspectral image (HSI) classification by exploiting local spectral-spatial information, but random train-test sampling from the same image can cause spatial patch overlap, leading to data leakage and optimistic performance estimates. This paper investigates same-class train-test spatial overlap in patch-based HSI classification using two measures: overlap percentage (OP), which quantifies the global amount of overlapped testing patch pixels, and average overlap ratio (AOR), which measures the local severity among affected testing patches. Experiments on the Pavia University dataset compare random and non-random spatial sampling using SVM, MLP, 2D-CNN, 3D-CNN, ViT, and MorpMamba. The results show that deep patch-based models achieve high accuracy under random sampling, with 3D-CNN reaching 96.17% Overall Accuracy (OA), but drop substantially under non-random spatial sampling, where 3D-CNN decreases to 55.20% and ViT and 2D-CNN drop by 40.71 and 38.81 percentage points (PP), respectively. Patch-size analysis further shows that increasing the patch size from 5x5 to 19x19 raises the random-sampling overlap percentage from 23.28% to 77.02%. These findings demonstrate that random patch-based evaluation can substantially inflate classification performance, especially for models that strongly exploit spatial context. The code associated with this paper is available at: https://github.com/mqalkhatib/Data_Leakage_in_HSI_Classification.

0.33

Evaluating Semantic and Spatial Guidance for Foundation Model Segmentation of Small-Scale PV in Remote Sensing Imagery

cs.CV ·2026-08-11 ·Roni Blushtein-Livnon, Tal Svoray, Osher Rafaeli, Michael Dorman, Itay Fischhendler +2 more
similarity 0.55
Abstract

Spatio-temporal PV data are essential for understanding adoption processes in off-grid regions, yet such data remain largely unavailable. Automated segmentation of remote sensing (RS) imagery offers a promising solution; yet, residential PV systems remain challenging targets because of their small size and sparse distribution, resulting in severe target-background imbalance. Vision-language foundation models (FMs) provide a data-efficient paradigm through prompt-based semantic and spatial guidance, but the relative contribution of different prompt types remains unclear. We systematically evaluate SAM3 for small-scale PV segmentation in RS imagery by comparing textual, geometric, and hybrid prompting, under varying supervision levels, training strategies, spatial resolutions, and imaging conditions. Multi-temporal aerial imagery from a large off-grid rural region serves as a study site, with findings validated across three additional datasets. Prompting strategy emerged as the dominant factor governing model behavior. Textual prompting consistently produced the lowest performance and showed the greatest sensitivity to supervision and imaging conditions. In contrast, spatial guidance substantially improved both segmentation accuracy and robustness. Hybrid prompting achieved the highest accuracy and stability, indicating that semantic and spatial guidance provide complementary information. Most performance gains were achieved with only a few hundred annotated samples, demonstrating strong data efficiency. Transfer learning had limited overall impact, with only modest improvements observed for textual prompting under limited supervision. Overall, our findings establish prompting strategy as a key determinant of SAM3 adaptation, robustness, and generalization, highlighting the potential of promptable FMs for scalable PV mapping in data-constrained off-grid regions.

0.04

Spectral-spatial vision test-time training for hyperspectral image classification

eess.IV cs.CV ·2026-11-02 ·Xiang Zhao
similarity 0.55
Abstract · 25 pages,9 figures,10 tables

Hyperspectral image (HSI) classification requires the representation of contiguous spectral signatures and spatial context under limited labeled supervision. Vision test-time training (TTT) offers an alternative to fixed-weight token mixing by performing self-supervised inner-model updates on the input sequence, but its application to center-pixel HSI classification remains insufficiently characterized. We present ViTTT-HSI, a spectral-spatial adaptation of the vision test-time training (ViT3) architecture. A spectral-aware embedding stem extracts ordered-band information using spectral convolution and forms spatial tokens without reducing their spatial resolution. Two ViT3-based TTT encoder blocks perform patch-conditioned token mixing, followed by the fusion of center-pixel and neighborhood-average features for classification. We evaluate the framework on Indian Pines, Pavia University, Salinas, and Houston 2013 using ten random seeds and dataset-specific training fractions of 1% or 5%. The mean overall accuracies are 97.6390%, 98.8753%, 99.3292%, and 98.4283%, respectively. Ablations reveal a substantial contribution from the TTT inner update on Indian Pines and a smaller positive contribution on Pavia University, while the relative benefits of the spectral stem and feature fusion depend on the dataset. The results demonstrate the feasibility of using patch-conditioned vision TTT for spectral--spatial HSI classification and highlight the role of the inner adaptation mechanism.

0.29