CS Paper Radar

Global-Local and Window-Concatenated Functional Random Forests: Early and Late Fusion of Window-Specific Functional Principal Components

Fabrizio Maturo, Annamaria Porreca

cs.LG stat.MLstat.ME ·submitted 2026-11-11 18:31 UTC · v1

Abstract

Functional random forests built on a single functional principal component decomposition of the whole domain may dilute features confined to short intervals, whose location is rarely known in advance. We propose two random forests based on window-specific functional principal components. The global-local functional random forest (GL-FRF) is a late-fusion method that stacks a tuned full-curve forest and one forest per window with penalised convex weights; it contains the full-curve forest as a feasible solution, and its weights indicate which parts of the domain drive the prediction. The window-concatenated functional random forest (WC-FRF) is an early-fusion method that trains a single tuned forest on the concatenated window-specific scores. On twenty UCR datasets, GL-FRF improved on a tuned full-curve forest with the same number of trees on 16, with Brier score reductions of up to 0.21, significantly after Holm adjustment. No method was best on more than six datasets. WC-FRF was more accurate than GL-FRF on 13 datasets but failed when the signal required the full curve, whereas GL-FRF had the smallest mean and maximum excess over the best method across all 32 simulated and real settings, which makes it the safer choice when the location of the signal is unknown.

Comment
36 pages, 12 figures, 9 tables

Closest papers

Nearest neighbours by meaning across the whole index.

Interpreting Random Forests: Comparing Surrogate Models and Rule Extraction Methods

cs.LG stat.MEstat.ML ·2026-11-14 ·Lea L. Mairhöfer, Jonathan B. Krause, Björn-Hergen Laabs, Silke Szymczak
benchmark similarity 0.59
Abstract · 4 figures

Random forests (RFs) often achieve high prediction performance, but their ensemble structure can be difficult to interpret. Surrogate models and rule extraction methods offer interpretable approximations, yet guidance on selecting an appropriate method for a specific research question remains limited. We compared 12 methods for interpreting RFs, with a decision tree as a prediction-oriented reference, across 11 simulation settings and 67 benchmark datasets for regression and classification. Evaluation addressed prediction and structural similarity to the RF, model complexity, prediction performance, and runtime. Simulations additionally assessed explainability using known effect and noise variables. No method performed best across all objectives. For regression, SLIM provided the strongest general performance for prediction similarity or performance on test data, while a linear model offered a simpler global interpretation of linear effects when performance was comparable. For classification, inTrees achieved the strongest overall benchmark performance, with CTree and OC-CART providing useful alternatives. ART consistently produced the most compact representations. NH and SIRUS offered strong structural similarity for regression, while MRT most closely reproduced RF variable usage in the classification simulations, illustrating that prediction-oriented rankings do not necessarily identify the best structural explanation. Outcome-type restrictions, runtime limits, and method-specific failures reduced result availability for several approaches and should be considered alongside performance when selecting a method. Method selection should therefore be guided by the outcome type, anticipated data structure, and intended interpretability objective. Prediction similarity alone is insufficient for explaining an RF and should be considered jointly with structural similarity and model complexity.

0.19

Tree species mapping in Denmark: A comparison of spectral-temporal features with geospatial foundation model embeddings

cs.CV cs.AIcs.LG ·2026-09-03 ·Alkiviadis Koukos, Spyros Kondylatos, Thomas Nord-Larsen, Lotte Nyborg, Christian Tøttrup +1 more
code/data similarity 0.59
Abstract · This preprint presents a national-scale tree species mapping framework for Denmark using Sentinel-1/2 time series, National Forest Inventory…

We map tree species across Denmark using National Forest Inventory plots and EO data, while evaluating the potential of foundation models for large-scale forest characterization. We compare two alternative input representations for tree species classification: (i) manually engineered spectral-temporal features (STF) derived from multi-temporal Sentinel-1 and Sentinel-2 observations, and (ii) embeddings generated by the EO FMs TESSERA and AlphaEarth. Both representations are complemented with canopy height information. Random forest, XGBoost, and Multi-Layer Perceptron (MLP) classifiers are evaluated for all input representations, with separate assessments for pure and mixed forest stands. The STF-based MLP achieves the highest classification performance, yielding macro F1 scores of 0.843 and 0.653 for pure and mixed stands, respectively. The MLP trained on TESSERA embeddings delivers competitive performance for pure stands, achieving results within 1.1 percentage points of the best-performing model. TESSERA consistently outperforms STF-based models when fewer than approximately 25% of training plots are available, demonstrating a substantial advantage under limited training data. Multi-year observations systematically improve classification accuracy relative to single-year inputs, while ablation experiments reveal the complementary contributions of Sentinel-1 backscatter, spectral indices, and canopy height data. The best-performing model is subsequently applied at the national scale to generate a 10 m tree species map of Denmark. Area-adjusted validation indicates an overall map accuracy of 79.9%. The resulting map, released as an open-access product, is the first high-resolution national tree species map of Denmark and provides a valuable resource for forest monitoring, ecological research, and land management applications.

0.30

Truncated Out-of-Bag Probability Gain: A Noise-Robust Criterion for Random-Forest Wrapper Feature Selection

cs.LG ·2026-10-21 ·Sejong Oh
benchmark similarity 0.59
Abstract

Random-Forest wrapper feature selection can use out-of-bag (OOB) predictions as a built-in validation signal, but the criteria that decide each forward step accumulate OOB estimation noise. We propose TOP-Gain (Truncated Out-of-bag Probability Gain), which scores a candidate feature by the mean sample-level increase in the OOB probability of the true class, with decreases truncated at zero. Under a simple noise model, truncation removes about two-thirds of the noise variance contributed by unaffected samples while preserving the signal, and it makes the ranking of candidate features markedly more reproducible. In a pre-registered benchmark on 26 OpenML-CC18 datasets, TOP-Gain achieved the best mean rank in area under the ROC curve (ROC-AUC), accuracy and Macro-F1 among 15 methods; it was significantly better than 13 of 14 baselines and matched the accuracy of VSURF with 21 times less single-core computation. Within an identical wrapper, it selected significantly more stable feature sets than net-gain, Brier, log-loss and accuracy criteria without loss of accuracy. A confirmatory study on 24 further datasets replicated the advantage over all baselines. The accuracy gains were broad rather than tied to specific dataset characteristics, and, as pre-registered, the stability gain grew as relevant features became sparser

0.28

Beyond Point Prediction: Artificial Representative Trees with Uncertainty

cs.LG stat.ML ·2026-09-21 ·Lea L. Mairhöfer, Silke Szymczak, Björn-Hergen Laabs, Tuwe Löfström-Cavallin
benchmark similarity 0.59
Abstract · 28 pages and 10 figures (without appendix)

Random forests (RFs) predict well but are opaque, whereas single decision trees are interpretable but unstable. Artificial representative trees (ARTs) were developed as interpretable surrogate models for RFs, but their use as standalone prediction models with uncertainty quantification has not been systematically investigated. We combine ARTs with leaf-wise Mondrian conformal predictive systems (CPS), enabling a single tree to provide continuous predictions, prediction intervals, and probabilities of exceeding arbitrary thresholds. We compared ARTs with CPS against decision trees with CPS and separate regression and probability trees across five simulation scenarios, 21 benchmark datasets, and a cross-sectional NHANES example data set. Repeated cross-validation assessed predictive performance, interpretability, and stability. ARTs with CPS yield compact, structurally stable trees with substantially more reproducible split-variable selection than decision trees across benchmark datasets and NHANES. Decision trees showed slightly better predictive performance and narrower prediction intervals, while coverage was broadly comparable. CPS-based trees generally achieved lower and less variable Brier scores than multi-model approaches. Combining ARTs with CPS therefore provides a single, interpretable, and stable model for continuous predictions and calibrated probabilities, balancing predictive performance with reproducibility and transparency in settings where stability and interpretability are essential.

0.26

Improving the Predictive Performance of Bootstrap Aggregating by Dirichlet Resampling

cs.LG stat.MLstat.ME ·2026-09-18 ·Quoc Viet Le, Joonha Park
similarity 0.58
Abstract · 29 pages (10 main text, 19 pages appendix), 21 tables, 3 algorithms. No figures

We revisit Breiman's observation that reducing inter-tree correlation without weakening individual trees can improve random forests. Building on this principle, we introduce two variants: Dirichlet-Multinomial Bagging Random Forest (DM) and Dirichlet-Weighted Random Forest (DW). Both modulate sample reweighting via a concentration parameter $α>0$. We provide a simple theoretical criterion that clarifies when these variants behave indistinguishably from standard random forests, and we use it to guide a lightweight tuning strategy. In a controlled evaluation on public classification benchmarks, DM and DW are consistently competitive and often stronger than other random-forest (RF) baselines, with negligible additional runtime.

0.33

Exploring Interpretability in Deep Learning-Based Forest Classification - Separating Similar Forest Classes using Sentinel-2 Time Series

cs.LG cs.AI ·2026-10-13 ·Maximilian Kirsch, Christine Preisach
similarity 0.57
Abstract

The ecological insurance theory posits that diverse forests are more resilient and better able to adapt to changing environmental conditions. Increasing drought and heat stress in Central Europe promotes mixed forests for maintaining ecosystem services. However, a diverse species distribution complicates remote sensing-based species mapping, as distinguishing pure from mixed species compositions at the pixel level remains challenging. We evaluated the potential of recurrent neural networks (RNNs) for tree-species classification from Sentinel-2 time series and investigated the spectral-temporal features underlying class discrimination, with particular emphasis on separating pure from mixed Norway spruce pixels. We compared RNN classification models with Random Forest (RF) and Extreme Gradient Boosting (XGBoost) and used Integrated Gradients (IG) to quantify feature attributions across spectral bands and time. The best-performing RNN achieved a Macro-F1 score of 0.851 on the test data, outperforming XGBoost by 0.012 and Random Forest by 0.103. Beyond classification performance, the attribution analysis revealed distinct spectral-temporal patterns associated with the separation of pure and mixed Norway spruce pixels. Attribution values increased particularly after approximately day of year (DOY) 125, while B11 showed the strongest contribution during periods of increasing spectral divergence between the two classes. These findings show that recurrent models can provide competitive classification performance while also revealing the phenological periods and spectral regions that are most relevant for distinguishing closely related forest compositions. Combining time-series classification with explainability therefore provides additional insight into the spectral-temporal mechanisms underlying tree-species discrimination and may support more detailed mapping of mixed forest structures.

0.05

What DNA Foundation Models Learn Beyond Sequence Composition

cs.LG cs.AI ·2026-10-11 ·Vincenzo Yuto Civale, Andrew David Bagdanov, Alberto Magi
top venue similarity 0.56
Abstract · Accepted to NeurIPS 2026 (Evaluations & Datasets Track). 9 pages of main text, 5 main figures, with references and appendices

Do genomic foundation models (FMs) provide predictive information beyond simple sequence composition? We evaluate frozen representations from five DNA FMs against normalized k-mer frequencies across 57 genomic classification datasets and two exploratory gene-expression regression tasks. Simple k-mer features remain competitive with most models, whereas NTv3 shows a consistent advantage; the ranking is robust to globally fixed and multi-scale k-mer baselines and alternative downstream probes. To test incremental predictive utility directly, we compare global 6-mer features with their concatenation with each FM embedding under a matched Random Forest protocol. NTv3 improves 46 of 57 datasets (mean $Δ$MCC = +0.072), while Caduceus provides smaller gains and the other models do not improve mean MCC. We separately decompose embeddings into a component linearly predictable from 6-mer frequencies and a ridge residual, finding that linear recoverability does not reliably indicate downstream utility. Exact 6-mer-preserving shuffles further reveal task-relevant sequence-order sensitivity, most pronounced for NTv3, although order sensitivity alone does not guarantee predictive improvements. Our results establish compositional features as essential controls and offer complementary diagnostics for evaluating the added value of frozen genomic representations, without making claims about fine-tuning capacity.

0.47

Compositional random forest

cs.LG stat.ML ·2026-11-10 ·Khaled Masoumifard, Stephan van der Westhuizen, Sugnet Gardner-Lubbe
similarity 0.55
Abstract

Compositional data have been analysed in the literature from three main points of view: directly on the simplex, in log-ratio coordinates, and as directional data on the unit sphere. In this paper, we construct a random forest framework for compositional responses that accommodates all three. Random forests are ensembles of decision trees, and the split criterion determines how each tree grows. We use the log-likelihood gain as the split criterion and consider distributions defined on the different spaces in which compositions can be represented. The Dirichlet likelihood has been studied in the literature, but its covariance is fixed by its mean up to a single scalar, and its maximum likelihood estimator has no closed form. To obtain more flexible criteria, we use two well-known transformations. First, the square-root transformation maps compositions onto the unit sphere, where existing directional distributions require numerical estimation. We introduce a tangent-normal (TN) distribution, which shares its parameters with the Kent distribution, and construct a split criterion whose gain is computed from additive sufficient statistics: the shape parameters are maximised out in closed form, up to an exponentially small truncation constant, so that no parameter is estimated iteratively. Second, the log-ratio transformation maps compositions to Euclidean space, where the multivariate logistic-normal distribution has a flexible covariance and a closed-form maximum log-likelihood, also based on additive sufficient statistics. Both criteria respond to a shift of the mean, measured relative to the covariance, and to a change in the covariance itself. In simulations, the most accurate forest depended on the covariance structure of the composition. The TN forest was the most accurate in two of the three designs.

0.28