CS Paper Radar

Subsurface schematisation from sparse CPT data using the tabular foundation model TabPFN

Taiga Saito, Fabian Campos Montero, Bruno Zuada Coelho, Ana Martins Teixeira, Stephen Wu, Timo Schweckendiek, Yu Otake

cs.LG cs.AI ·submitted 2026-11-11 22:57 UTC · v1

Abstract

Subsurface schematisation supports geotechnical decisions but remains challenging because sparse site-investigation data leave substantial gaps in information about ground conditions. This study explores TabPFN, a foundation model pretrained on general synthetic tabular data, for this task. Using only grid coordinates and soil behaviour type index observations from sparse cone penetration tests (CPTs), we first reconstruct two-dimensional sections without geotechnical fine-tuning or predefined layer boundaries. We evaluate the approach on synthetic sections and at two field sites in the Netherlands, comparing it with SchemaGAN, a conditional generative adversarial network trained on synthetic geotechnical sections, and with reference interpolation methods. In the baseline comparisons, TabPFN achieved mean errors close to SchemaGAN's on synthetic sections and lower mean errors at both field sites. On synthetic sections, TabPFN's mean absolute error decreased at each tested increase in CPT count, and, under the baseline input setting, its pointwise prediction intervals had mean coverage close to nominal levels. Subsequent fine-tuning on the synthetic sections used to train SchemaGAN improved synthetic test accuracy but increased mean errors at both field sites. These findings indicate that general-purpose tabular foundation models offer a promising route for updating subsurface predictions from sparse and evolving site-investigation data without retraining.

Closest papers

Nearest neighbours by meaning across the whole index.

From Synthetic Priors to Model Behavior: Structural Coverage in Tabular Foundation Models

cs.LG cs.AI ·2026-09-07 ·He Zhao, Ryan Thompson, Daniel M. Steinberg, Ashfaqur Rahman, Edwin V. Bonilla +1 more
benchmark similarity 0.61
Abstract

Tabular foundation models (TFMs) are commonly pretrained on large collections of procedurally generated synthetic tasks, yet it remains unclear how well these synthetic pretraining priors support the downstream tasks on which the models are evaluated. We study this question from a distribution-level attribution perspective. We recover or reconstruct the synthetic data generators of four TFMs and compare their generated tasks with datasets from two widely used tabular benchmarks. Each dataset is represented by a common set of structural descriptors capturing schema, feature distributions, dependence structure, response properties, and feature--response relationships. In this space, we measure how broadly and repeatedly each synthetic prior reaches benchmark tasks using structural coverage and normalized density, and examine whether stronger local support is associated with better predictive performance. We find substantial differences across synthetic pretraining priors: some generators provide consistently broader and denser support for benchmark tasks than others. Moreover, stronger synthetic-to-benchmark support is generally associated with better relative model performance. These results suggest that structural coverage provides a useful diagnostic for characterizing synthetic pretraining priors and relating their data-generating assumptions to downstream model behavior.

0.28

Scenario Generalization and Sector Transfer in Real Surface Irrigation Surrogate Modelling with Tabular Models

cs.LG ·2026-10-19 ·Alessandro Cagiano, M. Aybars Seyhan, Fabiola Gangi, Matteo Zignani, Daniele Masseroni
similarity 0.60
Abstract · Accepted at MACLEAN 2026 (Workshop on Machine Learning for Earth Observation), held in conjunction with ECML PKDD 2026

Surface irrigation planning requires screening many operating conditions and spatial configurations, while high-fidelity shallow-water-equation (SWE) simulations are expensive. We study tabular surrogate models for predicting maximum water height from drone-derived field morphology, terrain, hydrological, spatial, and scenario descriptors, where each row represents a spatial cell under one irrigation scenario. To assess deployment feasibility, we evaluate gradient-boosted trees, neural tabular models, AutoML, and tabular foundation models (TFMs) under two regimes: same-sector generalization, where a model predicts unseen scenarios after observing a limited set of configurations in the same sector, and sector-out transfer, where knowledge learned from source sectors is transferred to an unseen sector. The main findings are twofold: a) under the evaluated fixed 8/32 same-sector split, surrogates achieve strong performance, especially with raw spatial coordinates; b) sector-out transfer is substantially harder because the target sector has different coordinates and geometry. Sector-frame normalization of spatial coordinates improves transfer across similarly structured sectors, but does not solve cross-shape generalization. Overall, tabular surrogates appear useful for same-sector scenario screening under this fixed split, while robust transfer to unseen sectors remains open.

0.28

Probabilistic indirect models for undrained shear strength: addressing significant data missing and variability with advanced imputation and machine learning techniques

cs.LG cs.DB ·2026-08-14 ·Haibin Xiong, Shaoheng Dai, Peng Lan, Xuzhen He, Chenxi Tong +2 more
publishedhighly cited similarity 0.60
Abstract

Accurate prediction of undrained shear strength (su) is crucial for geotechnical design, but is often hampered by substantial uncertainty in traditional empirical methods. This study uses the CLAY/10/7490 global database to develop probabilistic indirect models to predict su based on Atterberg limits and piezocone cone penetration (CPTU) measurements. Firstly, the dataset has a high missing data rate and variability. We test three imputation methods - multivariate normal (MN), multiple imputation by chained equations (MICE), and miss forest (MF) - to fill the missing values. To validate their effectiveness, a Probabilistic Extreme Gradient Boosting (PXGB) model is developed, and the imputation methods are evaluated by comparing the PXGB's performance when trained on the imputed datasets against that on the original incomplete data. Secondly, the indirect model is built by integrating a multi-head attention (MHA) mechanism into an artificial neural network (ANN) to enhance information extraction from limited data, which leads to the MHA-based probabilistic neural networks (MHA-PNN) model. The models' performance, alongside a conventional MN-based prediction model, was evaluated using root mean square error (RMSE), coefficient of determination (R2), mean absolute percentage error (MAPE), conditional interval width (wCI), and coverage rate (CR). Results demonstrate that the proposed MN-enhanced MHA-PNN model substantially outperforms other models in both prediction accuracy and uncertainty quantification. These findings highlight the potential of this integrated strategy for building robust probabilistic indirect models in geotechnical applications, particularly when confronted with sparse and incomplete datasets.

0.61

Ranking Competing geologic interpretations via foundation-model-assisted generative hydrologic inversion

cs.LG ·2026-09-17 ·Harun Ur Rashid, Daniel O'Malley
benchmark similarity 0.59
Abstract

High-consequence subsurface decisions often rely on sparse data that permit competing geological interpretations. Determining consistency of these interpretations with the available observations remains challenging. We present a workflow that addresses this challenge by translating competing geologic interpretations into alternative priors and ranking them according to their consistency with hydraulic-head observations. A key step in this workflow is exploiting the broad knowledge of image-generation foundation models to transform nuanced geologic interpretations into data ready for computer modeling. For each interpretation, a text-to-image foundation model generates an ensemble of geologic images, and a separately trained variational autoencoder learns an interpretation-specific latent representation. A supervised inverse network maps head observations into this latent space, and the frozen decoder reconstructs an image that is mapped to a log-conductivity field. Steady-state flow simulations predict heads, and the aggregate normalized head error determines the ranking. We evaluate the framework using a synthetic benchmark based on the Johansen Formation with three interpretations of decreasing consistency with the reference geology. Across 595 test cases, the Precise \& Accurate interpretation produces lower normalized errors than Accurate in 58.5\% of cases and Mismatched in 82.5\% of cases. Accurate outperforms Mismatched in 65.5\% of cases. We then compare spatial representations of two published conceptual models of the Culebra Dolomite Member at the Waste Isolation Pilot Plant. The revised representation yields an aggregate normalized error of 7.598, compared with 8.595 for the original, consistent with the documented conceptual-model revision. The framework enables quantitative comparison of competing geological interpretations using available hydraulic observations.

0.29

RAD-TFM: Robust and Domain-Adapted Tabular Foundation Models

cs.LG cs.AI ·2026-11-06 ·Matthew Peroni, Franck Le, Vadim Sheinin
top venue similarity 0.58
Abstract · Accepted at NeurIPS 2026 (main track)

The development of tabular foundation models (TFMs) has accelerated in recent years, outperforming traditional ML methods on many tabular benchmarks. A key finding is that TFMs can be pretrained entirely on synthetic datasets, opening opportunities to design data generators that encourage desirable model properties. Prior work has mainly focused on crafting high-quality priors over generators to improve overall pretraining performance. Our key insight is that parameterizing the generator distribution enables an adversarial, distributional robustness perspective: during training, we can adapt the generator to emphasize datasets that are particularly challenging for the model, which we formalize by introducing an optimality gap measure. Further, we develop a method to align the synthetic data generation with real-world datasets from a given domain, constraining the adversary to generate realistic data. Together, these algorithms comprise the Robust and Domain-Adapted Tabular Foundation Models (RAD-TFM) pipeline, a model-agnostic adversarial training framework. Applied to the TabPFN V2 classifier, RAD-TFM improves performance across 6 diverse tabular benchmarks, with up to a 10.8 percentage-point improvement in mean normalized AUC over the strongest baseline, using fewer than 100k additional synthetic training datasets, less than 0.1% of the original pretraining data. Applying the same pipeline to TabICL V2 also improves performance across three major benchmarks, demonstrating the generality of the framework. These results highlight a promising new direction for targeted adversarial continued training of TFMs using synthetic data alone.

0.22

BRIDGE: Bridging Routine Inputs and Digital modelling for Geotechnical Engineering

cs.CE ·2026-10-02 ·Roshan Philip Saji, Waleed El-Sekelly, Tarek Abdoun, Mostafa E. Mobasher
similarity 0.57
Abstract

The mechanical properties needed for geotechnical design require laboratory tests and are thus available for far fewer field samples than routinely measured basic (index) properties which are available for nearly every sample. We term this difference as the characterization gap. Purely data-driven models do not explicitly enforce physical laws and may be unreliable beyond their training domain, while direct constitutive calibration preserves physical consistency but requires advanced laboratory data and expert judgment. We therefore propose BRIDGE (Bridging Routine Inputs and Digital modelling for Geotechnical Engineering) a modular framework that predicts mechanical properties from routinely available basic properties. BRIDGE screens the dataset for outliers and non-physical responses, bins the retained samples into Cells, and selects a small calibration dataset from each Cell, while the remaining samples are used for validation. A mechanical model is then calibrated for each calibration sample, and the converged calibrations collectively define the mechanical surrogate of the Cell together with its parameter ranges and the corresponding predictive envelope in the mechanical property space. A Cell is validated when most validation samples fall within this envelope, whereupon the surrogate can predict the mechanical property ranges of new samples whose basic properties lie within the Cell. In this work, we adopt a finite element implementation of the Extended Cam-Clay (ECC) constitutive model as the mechanical model, whose four parameters are calibrated in a multi-stage process. Applied to the Tokyo-CLAY/14/67760 database, BRIDGE predicts the mechanical properties of an entire Cell from calibrations of just 10 to 20% of its samples. Finally, since the mechanical model, the binning criteria, and the sampling method are components of BRIDGE, the framework is easily extensible.

0.25

Generating Heterogeneous 3D Geological Microstructures from 2D Images via a Stable Diffusion-Adversarial Model

cs.AI ·2026-09-17 ·Ali Aouf, Eric Laloy, Bart Rogiers, Christophe De Vleeschouwer
similarity 0.57
Abstract

Characterizing the physical properties of clay and cementitious materials matters across many fields, from materials science to geological waste disposal. Property simulation typically calls for 3D imaging, which is expensive, not always accessible, and technically limited for certain materials. Recent progress in deep generative models offers a way around this, reconstructing 3D volumes from the more easily acquired 2D images. Among GAN-based methods for 3D microstructure generation, SliceGAN has shown strong results for homogeneous isotropic and anisotropic systems. It struggles, however, to capture the finer detail of more complex heterogeneous microstructures, which motivates alternative generative frameworks. We introduce a hybrid approach that draws on the stability and generation quality of denoising diffusion models. Since no 3D ground truth is available, we replace the standard denoising loss with an adversarial loss, which yields a stable training process in our experiments. We show that the resulting model generates microstructures of varying complexity with minimal slice artefacts and close agreement with ground-truth phase fractions and structural descriptors.

0.17

Towards Reliable Scaling Laws for Tabular Foundation Models

cs.LG cs.AI ·2026-11-18 ·Johannes Hog, Neeratyoy Mallik, Steven Adriaensen, Jan Hendrik Metzen, Tom Viering +2 more
code/data similarity 0.56
Abstract

Scaling laws are widely used to predict foundation model performance and guide compute allocation. However, tabular foundation models (TFMs) differ from conventional foundation models in that they are pretrained on virtually unlimited synthetic data from configurable distributions, while ultimately being evaluated on real-world datasets. We investigate whether scaling laws for TabPFNv2-style models reliably extrapolate to larger compute budgets and generalize across evaluation settings. Our analysis reveals differences between synthetic and real-world scaling behavior and shows that accurately fitting a scaling law does not necessarily yield reliable prescriptions on unseen datasets. In particular, selecting configurations using too few evaluation datasets leads to systematically over-optimistic performance estimates. We further demonstrate that compute-optimal scaling requires expanding the range of synthetic pretraining dataset sizes alongside model size and training compute. Increasing the maximum numbers of rows and features sampled during pretraining improves performance and produces more predictable scaling trends. These insights guide the training of more efficient TabPFNv2-like models, using the open-source training pipeline of nanoTabPFN.

0.15