CS Paper Radar

Autoregressive Drillhole Modelling Under Distribution Shift

Yihao Ding, Daniel Yitian Su, Yiran Zhang, Christopher M. Gonzalez, Wei Liu

cs.LG ·submitted 2026-10-01 07:15 UTC · v1
benchmarkcode/data

Abstract

Autoregressive modelling has achieved remarkable success in language and sequence tasks by learning to predict future states from previous observation. Mineral-exploration drillholes provide a natural but largely unexplored setting for this paradigm: as drilling proceeds, lithology is revealed sequentially from shallow to deep, making prediction of deeper strata inherently autoregressive. Existing drillhole modelling, however, is dominated by spatial interpolation and reconstruction, or largely rely on masked modelling, leaving strictly autoregressive prediction largely underexplored. We introduce DrillBench, a benchmark of 49,671 Western Australian drillholes for next-layer prediction and autoregressive stratigraphic generation across a graded transfer spectrum, from local prediction through spatial shift to cross geological province transfer. Benchmarking classical, geostatistical, and neural models reveals a clear \emph{transfer boundary}: spatial and geochemical conditioning provides large local gains but deteriorates sharply under stronger shift, whereas lithology-sequence autoregressive models transfer more robustly. Guided by this finding, we develop a backbone-agnostic recipe combining large-scale pretraining on historical drillholes with spatial retrieval of neighbouring lithology. Retrieval is most effective in weathered cover, when local spatial continuity remains informative, whereas pretraining contributes more strongly in bedrock and under broader geological shift. Together, they retain strong local performance while improving generalisation under spatial and cross-province shift, most markedly on the most distant splits. The benchmark and code are available at https://github.com/yihaoding/drillbench.

Comment
work in progress

Closest papers

Nearest neighbours by meaning across the whole index.

Bayesian deep learning integration of geophysical and drilling data for 3D prediction of copper mineralization and drill targeting: a case study from the Kogodai prospect, Rudny Altai

cs.LG physics.geo-ph ·2026-09-08 ·Margarita Veshchezerova, Egor Barashov, Evgenii Gusev, Michael R. Perelshtein, Arlan Kasymzhan +2 more
similarity 0.68
Abstract · 28 pages, 14 figures, 2 tables. Submitted to Ore Geology Reviews

Exploration drill targeting in structurally complex terranes is hindered by sparse sampling, heterogeneous datasets, and the ambiguity of geophysical inversions. Here, we present an uncertainty-aware 3D workflow for the acceleration of time-to-discovery in brownfield explorations and apply it to the Kogodai prospect in the Rudny Altai metallogenic province. We jointly analyse existing drilling and geophysical data in a comprehensive approach, revealing hidden patterns in already available data. Drillholes and trenches were desurveyed to a common 3D reference frame, and assays were composited to a consistent spatial support to facilitate joint modelling with geophysical inputs. We develop Bayesian deep-learning models to predict 3D fields of Cu grade together with chargeability and apparent resistivity while quantifying epistemic uncertainty via Monte Carlo sampling. The original contribution of this work is to treat the problem not as pointwise regression between co-located observations, but as joint learning of spatially continuous 3D fields from sparse, heterogeneous exploration evidence. The resulting 3D predictions delineate a principal mineralized trend and several localized candidate zones that coincide with elevated induced polarization (IP) responses, while uncertainty mapping highlights where predictions are robust versus where additional drilling would be most informative. The continuous Cu-grade field can also be thresholded to produce binary prospectivity maps, allowing the sensitivity of target delineation to the chosen cutoff to be evaluated. The outputs are intended for qualitative interpretation and risk-aware drill targeting rather than resource estimation, and we discuss key limitations arising from incomplete provenance metadata for geophysical products and heterogeneity of historical sampling.

0.40

Ranking Competing geologic interpretations via foundation-model-assisted generative hydrologic inversion

cs.LG ·2026-09-17 ·Harun Ur Rashid, Daniel O'Malley
benchmark similarity 0.67
Abstract

High-consequence subsurface decisions often rely on sparse data that permit competing geological interpretations. Determining consistency of these interpretations with the available observations remains challenging. We present a workflow that addresses this challenge by translating competing geologic interpretations into alternative priors and ranking them according to their consistency with hydraulic-head observations. A key step in this workflow is exploiting the broad knowledge of image-generation foundation models to transform nuanced geologic interpretations into data ready for computer modeling. For each interpretation, a text-to-image foundation model generates an ensemble of geologic images, and a separately trained variational autoencoder learns an interpretation-specific latent representation. A supervised inverse network maps head observations into this latent space, and the frozen decoder reconstructs an image that is mapped to a log-conductivity field. Steady-state flow simulations predict heads, and the aggregate normalized head error determines the ranking. We evaluate the framework using a synthetic benchmark based on the Johansen Formation with three interpretations of decreasing consistency with the reference geology. Across 595 test cases, the Precise \& Accurate interpretation produces lower normalized errors than Accurate in 58.5\% of cases and Mismatched in 82.5\% of cases. Accurate outperforms Mismatched in 65.5\% of cases. We then compare spatial representations of two published conceptual models of the Culebra Dolomite Member at the Waste Isolation Pilot Plant. The revised representation yields an aggregate normalized error of 7.598, compared with 8.595 for the original, consistent with the documented conceptual-model revision. The framework enables quantitative comparison of competing geological interpretations using available hydraulic observations.

0.29

Simulated petrological labels enable probabilistic mineralogy prediction from whole-rock geochemistry

cs.LG ·2026-10-27 ·Matthew J. Liu, Aaron D. Kaplan, Michael L. Whittaker, Kristin A. Persson
similarity 0.62
Abstract

Modal mineralogy, the mineral identities and abundances in rocks, guides geological studies and industrial resource extraction. While measuring modal mineralogy is time-intensive, whole-rock chemical measurements furnish elemental concentrations at higher throughput. Predicting mineralogy from elemental data, however, poses a one-to-many inverse problem, and labeled mineralogical datasets are too sparse to train on at scale. We address both challenges by simulating petrological labels that express multiple formation conditions per composition and at a scale unattainable by measurements. Nearly one million simulated labels supervise the middle component of a three-part model: a transformer encoder self-supervised on roughly 788,000 whole-rock compositions, the simulation-trained heads that predict distributions over mineral assemblages, and a recalibration layer that aligns the predictions with measured mineralogy. Zero-shot on 640 volcanic rocks, the model discriminates the presence of most mineral categories and predicts which rocks are richer in each mineral. However, the predicted abundances are systematically offset from measured values. Recalibration fit on as few as 25 labeled rocks recovers 76 percent of the likelihood improvement from fitting on the full 368-rock fitting set, while abundance error responds more slowly (11.0% zero-shot and 9.5% with the full fitting set). This work demonstrates how a machine learning model in the geochemical sciences can both leverage an imperfect simulator and mitigate its systematic biases.

0.26

GeoCFM: Positive-Only Conditional Flow Matching for Mineral Occurrence Sampling

cs.LG ·2026-09-28 ·Moshe Eliasof, Eldad Haber
top venuebenchmark similarity 0.62
Abstract · ECCV 2026

Critical mineral discovery is a positive-only problem: deposits are observed as sparse locations, while unlabeled regions are not reliable negatives, and similar geophysical signatures can arise from different subsurface states. We therefore model mineral targeting as learning a conditional spatial distribution over occurrence locations, $π(p\mid d)$, given geo-images $d$, rather than predicting a deterministic per-pixel score map. We introduce GeoCFM, a conditional flow-matching model that generates mineral occurrence point sets conditioned on multi-channel geo-images; GeoCFM learns a point-wise transport field in $\mathbb{R}^2$, using UNet features with point-conditioned velocity prediction to bridge dense rasters and sparse supervision without pseudo-negatives. On a synthetic magnetics--geochemistry benchmark with latent activation and on USGS Earth MRI data with a spatially disjoint tile split, GeoCFM improves geometric agreement with observed occurrences over score-map and non-conditional baselines, while representing epistemic uncertainty through conditional sampling.

0.43

An AI-assisted conditioning and geological interpretation workflow for usage in implicit geological modeling

cs.AI physics.geo-phcs.LG ·2026-10-07 ·Stefan Carpentier, Jan Diederik van Wees, Eva de Boever, Jan Niederau, Camille Chapeland +3 more
similarity 0.60
Abstract · 17 page, 19 figures

Implicit modeling and Relative Geologic Time are geological modeling techniques that enable more efficient, faster, less biased and more reproducible modeling results. For optimal operation, these techniques require many well-constrained input data. In the framework of the Horizon Europe GO-Forward and MOOI WarmingUP GOO projects and to accelerate Implicit modeling, Machine Learning (ML) methods have been tested and implemented in a toolkit for the interpretation of (onshore) seismic data from the shallow to deep range (+- 300 - 3500 m). The goal is to rapidly characterise this depth domain by efficient interpretation of horizons and faults in seismic data. The first step is to improve the signal by applying AI techniques like self-supervised and semi-supervised contrastive learning CNN's for noise reduction and interpolation. Next, horizons and faults are interpreted with minimal use of human-generated training data by using (semi-) self-supervised methods. The resulting developed toolkit supports the application of the implemented algorithms in an efficient workflow. As a first demonstration, the top of the Dutch Maassluis Formation has been interpreted in the Leeuwarden and Waalwijk 3D seismic cubes. Overall, this study demonstrates that AI-assisted interpretation workflows have reached a level of maturity that allows their integration into applied geological modeling and decision-making.

0.22

Expert lithological labels do not improve learning when used to predict marine sediment properties

cs.AI physics.geo-ph ·2026-10-26 ·John M. Aiken, William Gilpin, Thorsten W. Becker
similarity 0.60
Abstract

Scientific drilling has produced large archives of physical, chemical and magnetic measurements linked to descriptions of the rock recovered. These archives are mostly incomplete: instruments sample at different intervals, recovery is partial, and destructive analyses preclude later measurement of the same material. We train an self-supervised embedding model on the LILY ocean drilling database using six routinely logged properties (bulk density, magnetic susceptibility, natural gamma radiation, and red, green and blue reflectance) across 236,963 observations from 296 boreholes, with a masked variational autoencoder that has a heteroscedastic Gaussian decoder and learnable mask tokens. The embeddings separate geologist-assigned lithologies as well as a variant trained with those labels, and increasing the weight on the labels degrades reconstruction while leaving classification unchanged. The labels are recoverable from the measurements geologists use to assign them, so training on them adds nothing. Lithological categories compress continuous measurements into classes, and compressing them discards what the measurements record about the processes that produced the rock. The embeddings also predict physical properties that were absent from training, on boreholes the model has never seen. While depth should account for this since sediments compact with burial, it does not. On held-out boreholes depth alone reaches a median $R^2$ of -0.02, against 0.20 for the embedding.

0.32

Basin2Vec: Learning Basin-Centric Embeddings for Scalable Hydrological Modeling

cs.LG cs.AI ·2026-11-03 ·Muhammad Talha, Xiaofeng Liu, Yuan Yang, Pang-Ning Tan, A. Pouyan Nejadhashemi
code/data similarity 0.59
Abstract

Reliable streamflow prediction across time and in ungauged basins requires representations that capture hydrologically relevant basin characteristics. Existing methods rely on either fixed handcrafted attributes or general-purpose Earth observation embeddings not tailored to basin structure. We introduce \textbf{Basin2Vec}, a self-supervised framework that learns compact basin representations from hydroclimatic and physiographic data without observed streamflow. Basin2Vec organizes representation learning around basin scale and relationships across space and time. Across multiple forecasting architectures, it consistently outperforms predefined attributes and general-purpose geospatial embeddings in temporal generalization and zero-shot spatial transfer. Basin2Vec provides average relative gains of up to 8.3% temporally and achieves 3.5x the median predictive performance of the strongest baselines on unseen basins. These results demonstrate the effectiveness of basin-centric pretraining for transferable hydrologic prediction. Code is available at https://github.com/Muhammad-Talha-MT/basin2vec

0.32

Well-Log Reconstruction with Pre-trained Geological Knowledge: A Knowledge-Conditioned Large-Language-Model Generative Agent

cs.LG cs.AI ·2026-11-14 ·Yongan Zhang, Yitao Yu, Yizheng Li, An Gong, Youzhuang Sun
benchmark similarity 0.59
Abstract · 35 pages

Missing well-log intervals make subsurface reconstruction ill-posed. Existing numerical regression models lack geological prior knowledge to constrain ambiguous inverse mappings. We propose GeoLog-Agent, a knowledge-conditioned generative agent using pre-trained large language models (LLMs) as a source of geological priors. A universal Geo-Harness translates LLM stratigraphic reasoning into formation-anchored petrophysical trajectories, feature-wise linear modulation, and conditional adversarial constraints. On the Norwegian North Sea benchmark, GeoLog-Agent achieves a mean $R^2$ of 0.9307 and a peak $R^2$ of 0.9713. It exceeds the nine-baseline average $R^2$ of 0.8387 by 0.0920, with a 30.97% average RMSE reduction, and exceeds the strongest baseline by 0.0274--0.0498 in $R^2$, with 16.25%--24.05% lower RMSE. Ablations confirm the effectiveness and synergy of geological prior reasoning, multi-scale dilated generation, and conditional adversarial modules. Transferring the coupling mechanism to six conventional deep learning models improves their $R^2$ by 0.0162--0.0789 with cross-well stability. These results demonstrate how foundation-model domain reasoning can constrain deep generative reconstruction of missing petrophysical measurements.

0.33