CS Paper Radar

Random Jump Forests

Martin Bladt, Rasmus Frigaard Lemvig

cs.LG stat.MEstat.ML ·submitted 2026-10-23 11:15 UTC · v1
benchmarktheory

Abstract

Random jump forests are introduced as a nonparametric tool for estimating conditional cumulative transition rates and occupation probabilities in multi-state models. The main idea is that a single tree partitions the covariate space for all transitions simultaneously, using partially observed trajectories, and with Nelson--Aalen estimators at the terminal nodes. Occupation probabilities are recovered through the product integral without requiring the Markov assumption. Uniform consistency and a functional central limit theorem for the estimators of cumulative rates can be established under the assumption that the covariate vector takes finitely many values, and under suitable splitting conditions. In particular, pointwise confidence intervals and simultaneous bands are available in the Markov case. Simulation studies show a predictive accuracy close to the correctly specified benchmark models and marked improvements when benchmarks are misspecified, with further illustration on four biomedical data sets. In the coverage study, confidence bands attain near-nominal coverage in several settings, with pooling of heterogeneous rates being the main source of possible undercoverage. The method, including prediction assessment and variable importance, is implemented in the R package JumpForests.

Comment
85 pages, 19 figures and 10 tables

Closest papers

Nearest neighbours by meaning across the whole index.

Random Hazard Forests

cs.LG stat.MLstat.ME ·2026-08-21 ·Hemant Ishwaran, Eileen M. Hsich, Udaya B. Kogalur, Donald K. K. Lee
similarity 0.58
Abstract

Clinical data sources such as electronic health records and wearable sensors record patient status repeatedly over follow-up, often at irregular times and on different schedules for different measurements. These data create opportunities for continuously updated, individualized risk prediction. Existing approaches, however, often simplify the temporal structure for model fitting. We introduce Random Hazard Forests (RHF), a survival tree ensemble that estimates how a patient's hazard changes in continuous time as new measurements become available. The method formulates the estimation problem directly through a nonparametric hazard likelihood for predictable covariate processes. An efficient working model guides tree construction, after which flexible time-varying hazards are estimated for each terminal node. Given any predictable covariate path, each tree follows the path through its terminal nodes over time and assembles the corresponding node-level hazards into a trajectory. Averaging these trajectories across trees yields the pathwise hazard estimate. Because routing at each time uses only the covariate state available immediately beforehand, the construction accommodates internal longitudinal covariates without lookahead. Simulations and an intensive care application show that the forest accurately estimates changing risk under irregular and asynchronous covariate updates.

0.31

Learning Transition Kernels of Jump-Diffusion Processes with Conditional Diffusion Models

cs.LG stat.ML ·2026-10-06 ·Yuzhen Zhao, Yating Liu, Quentin Guibert
benchmarkhot direction similarity 0.56
Abstract

We study the problem of learning transition kernels for time-homogeneous jump-diffusion processes using conditional diffusion models, with the goal of generating new sample paths from training data consisting of N independent trajectories observed on a high-frequency discrete time grid. On the theoretical side, we establish non-asymptotic bounds for the conditional score estimation error and for the KL divergence between the laws of the true and generated discretely observed paths. On the numerical side, we first evaluate our method on synthetic data to assess the theoretical findings and benchmark its performance against the approach of Gao et al. (2025). We then apply our method to real-world data and investigate its performance on a probabilistic forecasting task.

0.37

Dirichlet Process Mixtures of Trees with Gaussian Process Splits: A Bayesian Nonparametric Framework with Posterior Contraction Rate

cs.LG stat.MEmath.STstat.ML ·2026-08-22 ·Subhasish Basak, Anik Roy, Sourabh Bhattacharya
benchmarktheory similarity 0.54
Abstract · Comments and questions welcome!

We propose a Bayesian nonparametric mixture of regression trees with a Dirichlet process prior over tree-parameter pairs, enabling data-driven selection of ensemble size and unifying CART, BART, random forests, and boosting. A novel splitting rule driven by the posterior predictive of a Gaussian process within each terminal node generates flexible, smooth decision boundaries; remarkably, the GP density cancels exactly in the Metropolis--Hastings ratio for GROW/PRUNE moves, ensuring computational feasibility. An exact Gibbs sampler for posterior predictive inference propagates uncertainty through random tree traversal. A parallel MPI implementation distributes independent tree updates across processors, achieving adequate speedups. We prove posterior consistency at rate $n^{-1/4}$ in Hellinger distance under only continuity of the true regression function, allowing misspecification, via the identity $h(Θ)=0$. Simulations on Friedman benchmark show near-nominal coverage (0.94 Gaussian, 0.92 Cauchy), robust to high-dimensional noise and heavy tails, outperforming BART and bagged CART. Applications to QSAR toxicity, crime, riboflavin, wheat genomics, and air quality confirm reliable credible intervals and automatic sparsity. The DP mixture offers a principled, robust, theoretically justified alternative for challenging regression with honest uncertainty quantification.

0.32

Martingale Posterior Cox Processes

cs.LG stat.MLstat.ME ·2026-10-17 ·Joshua Rooijakkers, Edwin Fong
similarity 0.51
Abstract

Cox processes provide a principled Bayesian nonparametric framework for modeling event-based data through an underlying intensity function. However, existing inference methods for estimating the intensity typically rely on computationally expensive Markov chain Monte Carlo sampling or approximate inference techniques. In addition, to ensure tractability, these methods often define priors over indirect latent structures, making them challenging to interpret. In this paper, we introduce the Martingale Posterior Cox Process (MPCP), which recasts Bayesian intensity estimation as inference over unobserved events given the observed event history. This perspective avoids the need to define complex priors while retaining the flexibility of nonparametric modeling. We propose a scalable, GPU-friendly copula-based predictive update, and provide strong theoretical guarantees for the convergence of the predictive resampling algorithm. Empirically, we show that MPCP provides excellent performance in both simulations and real data at a fraction of the computational cost of competing Bayesian approaches.

0.24

Anytime Valid Confidence Bands for Distribution Functions

cs.LG stat.ML ·2026-10-26 ·Bror Hultberg, Dave Zachariah, Antônio H. Ribeiro
similarity 0.51
Abstract

Nonparametric inference often requires characterizing an unknown function, such as a cumulative distribution function (CDF), across its entire domain rather than a specific point. We develop confidence bands for distribution functions of a data stream that are valid simultaneously over all times and all points, even when each observation may depend arbitrarily on the past. We begin by deriving such bands for the time-averaged conditional CDF and show how their widths are determined by the depth of a binary search tree. We then show that this result is an instance of a framework through which one can construct valid confidence bands uniformly over time and over an arbitrary index set for any indexed family of time-averaged distribution functions. We illustrate the framework on importance-weighted and right-censored CDFs, as well as multivariate CDFs.

0.26

Prediction-powered inference for time series across space

cs.LG stat.MEstat.ML ·2026-10-06 ·Shahzar Rizvi, David Burt, Vishwak Srinivasan, Renato Berlinghieri, Stefano Del Col +1 more
publishedcode/data similarity 0.50
Abstract · Accepted to TS-LIMITS Workshop at NeurIPS 2026

The following motif is common in spatiotemporal settings: we have a sequence of covariate and label pairs observed for a relatively short, recent time period. We have access to unlabeled covariates over a longer time period. Data is observed over many spatial locations. For instance, crop yield might be observed over a large geographical area for recent years, but weather data (which is informative about crop yield) is available for a much longer period. The goal is to estimate, at each spatial location, the expected label (e.g., crop yield) in the future and provide a valid confidence interval for this value. The observed time period alone is too short for reliable estimates. Imputing missing labels with machine learning can cause substantial bias. Prediction-powered inference (PPI) can correct for this bias, but it relies on an i.i.d. assumption that breaks under our expected temporal dependencies. Heteroskedasticity and autocorrelation consistent (HAC) procedures account for temporal correlation, but have not been adapted to cases where some labels are imputed. We provide reliable point estimates and confidence intervals given: short labeled time series (across spatial locations), a longer unlabeled time series, and an imperfect predictor of labels given covariates. We show our method outperforms natural alternatives.

0.47

CRNDiff: Count-Native Diffusion Framework via Chemical Reaction Networks

cs.LG ·2026-09-25 ·Yuxuan Qiu, Praful Gagrani, Tetsuya J Kobayashi
similarity 0.50
Abstract · 28 pages, 9 figures, 10 tables. Under review as a conference paper at ICLR 2027

Scientific measurements such as single-cell RNA (scRNA) sequencing often take the form of nonnegative integer counts, whereas continuous-state diffusion models approximate this discrete structure using continuous coordinates. Building on stochastic chemical reaction networks (CRNs), a class of count-native Markov jump processes, we introduce CRNDiff, a structured framework that combines count-space diffusion with inference-time conditioning on rare subpopulations. An independent birth--death instantiation yields a closed-form transition kernel for forward noising. This kernel enables reverse sampling via forward-filtering backward-sampling (FFBS) and supports data-driven selection of the terminal noising time, eliminating the need for a validation sweep. This tractability also lets us introduce tilted Feynman--Kac (FK) steering, a method for sampling target subpopulations from a frozen generator without retraining. By tilting posterior marginals before FK particle correction, steering mitigates importance-weight concentration when the target population is rare. Using scRNA-seq data from the human heart cell atlas, we test the ability of CRNDiff to generate cell-type-specific distributions. Across the three evaluated target populations, CRNDiff achieves the highest conditional fidelity among the evaluated generative models, with larger mean purity margins for rarer target populations. Generated cells preserve marker-level differential-expression structure. Replacing real training cells for the target classes with generated cells yields downstream classification performance approaching that of the real-data reference.

0.26

TreeWalker: Partial Evaluation for Grouped Tree-Ensemble Inference

cs.LG cs.PFcs.PL ·2026-10-02 ·Durmus Karatay, Richard Newman
top venuebenchmarktheorycode/data similarity 0.50
Abstract · 23 pages, 9 figures, 11 tables. Accepted at NeurIPS 2026. Code and data: https://github.com/Shopify/treewalker/tree/neurips2026-final

Many prediction tasks evaluate a trained tree ensemble on groups of rows that share feature values: discrete-time survival models expand each patient into $G$ time steps, click-through-rate models score every item in a search session, and scenario analyses vary a few inputs while holding the rest fixed. Standard inference treats each row independently and ignores this input-side correlation. We present TreeWalker, which instantiates partial evaluation for grouped tree-ensemble inference: constant features are static, varying features dynamic. TreeWalker walks each tree once per group, partitions a row bitmask at varying splits, and skips empty subtrees via an unsplit shortcut. It reads standard LightGBM and XGBoost models; training is unchanged. We prove a structural work decomposition: per-tree work splits into the constant-projected subtree size $|T_c|$, $G$ leaf writes, and a predicate-mask provisioning cost $Q$. For the trace evaluator, per-row work approaches $(d_v+1)/(d+1)$ as $G \to \infty$, giving asymptotic speedup $1/(1-\barρ_G)$. On Intel, TreeWalker is ${\sim}3\times$ faster than a row-independent traversal at the reference model size ($T{=}500$, $L{=}8$; $G{=}16$ for SUPPORT/FLCHAIN and empirical groups for Expedia) and $7.7$--$9.1\times$ faster at $G{=}128$ on the survival datasets; on Arm, $2.5$--$2.9\times$ and $5.5$--$6.0\times$. On a scenario-analysis benchmark it is faster in all $16$ configurations on both machines. For f64 models, outputs match treelite GTIL up to tree-ordering roundoff; for f32 models, TreeWalker's f64 leaf accumulation is closer than native f32 to an exactly rounded reference on every audited row.

0.53