CS Paper Radar

Expert lithological labels do not improve learning when used to predict marine sediment properties

John M. Aiken, William Gilpin, Thorsten W. Becker

cs.AI physics.geo-ph ·submitted 2026-10-26 23:43 UTC · v2

Abstract

Scientific drilling has produced large archives of physical, chemical and magnetic measurements linked to descriptions of the rock recovered. These archives are mostly incomplete: instruments sample at different intervals, recovery is partial, and destructive analyses preclude later measurement of the same material. We train an self-supervised embedding model on the LILY ocean drilling database using six routinely logged properties (bulk density, magnetic susceptibility, natural gamma radiation, and red, green and blue reflectance) across 236,963 observations from 296 boreholes, with a masked variational autoencoder that has a heteroscedastic Gaussian decoder and learnable mask tokens. The embeddings separate geologist-assigned lithologies as well as a variant trained with those labels, and increasing the weight on the labels degrades reconstruction while leaving classification unchanged. The labels are recoverable from the measurements geologists use to assign them, so training on them adds nothing. Lithological categories compress continuous measurements into classes, and compressing them discards what the measurements record about the processes that produced the rock. The embeddings also predict physical properties that were absent from training, on boreholes the model has never seen. While depth should account for this since sediments compact with burial, it does not. On held-out boreholes depth alone reaches a median $R^2$ of -0.02, against 0.20 for the embedding.

Closest papers

Nearest neighbours by meaning across the whole index.

Simulated petrological labels enable probabilistic mineralogy prediction from whole-rock geochemistry

cs.LG ·2026-10-27 ·Matthew J. Liu, Aaron D. Kaplan, Michael L. Whittaker, Kristin A. Persson
similarity 0.68
Abstract

Modal mineralogy, the mineral identities and abundances in rocks, guides geological studies and industrial resource extraction. While measuring modal mineralogy is time-intensive, whole-rock chemical measurements furnish elemental concentrations at higher throughput. Predicting mineralogy from elemental data, however, poses a one-to-many inverse problem, and labeled mineralogical datasets are too sparse to train on at scale. We address both challenges by simulating petrological labels that express multiple formation conditions per composition and at a scale unattainable by measurements. Nearly one million simulated labels supervise the middle component of a three-part model: a transformer encoder self-supervised on roughly 788,000 whole-rock compositions, the simulation-trained heads that predict distributions over mineral assemblages, and a recalibration layer that aligns the predictions with measured mineralogy. Zero-shot on 640 volcanic rocks, the model discriminates the presence of most mineral categories and predicts which rocks are richer in each mineral. However, the predicted abundances are systematically offset from measured values. Recalibration fit on as few as 25 labeled rocks recovers 76 percent of the likelihood improvement from fitting on the full 368-rock fitting set, while abundance error responds more slowly (11.0% zero-shot and 9.5% with the full fitting set). This work demonstrates how a machine learning model in the geochemical sciences can both leverage an imperfect simulator and mitigate its systematic biases.

0.26

Ranking Competing geologic interpretations via foundation-model-assisted generative hydrologic inversion

cs.LG ·2026-09-17 ·Harun Ur Rashid, Daniel O'Malley
benchmark similarity 0.62
Abstract

High-consequence subsurface decisions often rely on sparse data that permit competing geological interpretations. Determining consistency of these interpretations with the available observations remains challenging. We present a workflow that addresses this challenge by translating competing geologic interpretations into alternative priors and ranking them according to their consistency with hydraulic-head observations. A key step in this workflow is exploiting the broad knowledge of image-generation foundation models to transform nuanced geologic interpretations into data ready for computer modeling. For each interpretation, a text-to-image foundation model generates an ensemble of geologic images, and a separately trained variational autoencoder learns an interpretation-specific latent representation. A supervised inverse network maps head observations into this latent space, and the frozen decoder reconstructs an image that is mapped to a log-conductivity field. Steady-state flow simulations predict heads, and the aggregate normalized head error determines the ranking. We evaluate the framework using a synthetic benchmark based on the Johansen Formation with three interpretations of decreasing consistency with the reference geology. Across 595 test cases, the Precise \& Accurate interpretation produces lower normalized errors than Accurate in 58.5\% of cases and Mismatched in 82.5\% of cases. Accurate outperforms Mismatched in 65.5\% of cases. We then compare spatial representations of two published conceptual models of the Culebra Dolomite Member at the Waste Isolation Pilot Plant. The revised representation yields an aggregate normalized error of 7.598, compared with 8.595 for the original, consistent with the documented conceptual-model revision. The framework enables quantitative comparison of competing geological interpretations using available hydraulic observations.

0.29

Autoregressive Drillhole Modelling Under Distribution Shift

cs.LG ·2026-10-01 ·Yihao Ding, Daniel Yitian Su, Yiran Zhang, Christopher M. Gonzalez, Wei Liu
benchmarkcode/data similarity 0.60
Abstract · work in progress

Autoregressive modelling has achieved remarkable success in language and sequence tasks by learning to predict future states from previous observation. Mineral-exploration drillholes provide a natural but largely unexplored setting for this paradigm: as drilling proceeds, lithology is revealed sequentially from shallow to deep, making prediction of deeper strata inherently autoregressive. Existing drillhole modelling, however, is dominated by spatial interpolation and reconstruction, or largely rely on masked modelling, leaving strictly autoregressive prediction largely underexplored. We introduce DrillBench, a benchmark of 49,671 Western Australian drillholes for next-layer prediction and autoregressive stratigraphic generation across a graded transfer spectrum, from local prediction through spatial shift to cross geological province transfer. Benchmarking classical, geostatistical, and neural models reveals a clear \emph{transfer boundary}: spatial and geochemical conditioning provides large local gains but deteriorates sharply under stronger shift, whereas lithology-sequence autoregressive models transfer more robustly. Guided by this finding, we develop a backbone-agnostic recipe combining large-scale pretraining on historical drillholes with spatial retrieval of neighbouring lithology. Retrieval is most effective in weathered cover, when local spatial continuity remains informative, whereas pretraining contributes more strongly in bedrock and under broader geological shift. Together, they retain strong local performance while improving generalisation under spatial and cross-province shift, most markedly on the most distant splits. The benchmark and code are available at https://github.com/yihaoding/drillbench.

0.33

DARWIN: a Geospatial Foundation Model for Marine Intelligence

cs.AI physics.ao-ph ·2026-11-17 ·Coen van den Elsen, Laurens Geffert
similarity 0.59
Abstract

We present DARWIN, a geospatial foundation model for marine intelligence. Recent Earth-observation foundation models have concentrated on terrestrial domains. The marine realm poses distinct challenges: missingness, multi-modal physical and biogeochemical variables, sparse labels, and dynamics that are transient, multi-scale, and non stationary over time. DARWIN adapts a masked-autoencoder backbone to these properties through missingness-aware encoding, expanded spatial and temporal masking, and Fourier-style positional embeddings, and is pretrained on 26 years of global Copernicus reanalysis together with GEBCO bathymetry. On predicting chlorophyll-a, the pretrained backbone attains the lowest monthly relative error in every month of 2022, outperforming both no-encoder baselines and an identically architected random-init encoder. At 10% of the supervised training data, DARWIN already surpasses the random-init model trained on 100% of the data. Embedding analysis and qualitative reconstructions show that DARWIN captures coherent oceanographic structure rather than memorising input statistics, providing a scalable backbone for downstream marine applications such as anomaly detection, species distribution modelling, and environmental monitoring.

0.25

Many Pixels, One Composition: Self-Supervised Spectra for Specimen-Level Prediction

cs.CV cs.LG ·2026-11-04 ·Eleftheria Tetoula-Tsonga, George Arvanitakis, Theodoros Giannakas
similarity 0.59
Abstract

Can hyperspectral (HS) imaging be used to predict the elemental composition of physical specimens found on the Earth's surface? We approach this problem as a supervised regression task, using an open dataset of specimens, each comprising a HS cube and elemental characterization. In particular, we train an encoder in a self-supervised fashion that operates on pixel-level; then freeze the encoder and train a lightweight attention-based regression head in a supervised manner, by pairing many representations of a given specimen with its corresponding elemental label. In test specimens, our approach significantly outperforms traditional spectral matching and unmixing baselines. It also outperforms supervised learning models based on the mean spectrum (of similar model capacity), while full model fine-tuning achieves the strongest performance, at substantially higher model capacity.

0.10

MarineSet: Large-Scale Underwater Acoustic Data Curation and evaluation

cs.SD eess.AS ·2026-10-21 ·Hilde I. Hummel, Sandjai Bhulai, Rob D. van der Mei, Burooj Ghani
similarity 0.58
Abstract

The scarcity of labeled underwater acoustic data limits the development of data-driven models for marine acoustic monitoring, motivating the use of self-supervised learning to exploit the abundance of unlabeled recordings. However, large-scale data alone does not guarantee effective representation learning, because model performance is strongly influenced by the distribution of the training data. To facilitate such training data, this study introduces a scalable curation pipeline that curates 3,000 hours from 41 years of raw NOAA hydrophone recordings, producing MarineSet, a large-scale and diverse dataset designed for underwater acoustic learning. A baseline model, MarineNet, trained on MarineSet achieves competitive performance on marine mammal call and ship-type classification, highlighting the value of systematic data curation for developing self-supervised learning models in underwater acoustics.

0.20

BenthicDINO: Physics-Informed Self-Distillation for View-Invariant Side-Scan Sonar Representations

cs.CV cs.AIcs.LG ·2026-08-24 ·Taqi Hamoda, Hayat Rajani, Nuno Gracias
benchmarkhot direction similarity 0.57
Abstract

Automated interpretation of side-scan sonar (SSS) imagery is traditionally hindered by view-dependence, as recorded intensities inextricably conflate intrinsic seabed reflectivity with transient acquisition geometries and acoustic artifacts. To address this, we introduce BenthicDINO, a physics-informed self-supervised learning framework that decouples acquisition dynamics from seabed representations. Utilizing a ConvNeXt-v2 backbone, physics-guided augmentations, and an HSIC penalty, BenthicDINO explicitly enforces feature invariance to physical viewing parameters like slant range and incidence angle. Evaluations demonstrate exceptional data efficiency - achieving near-peak performance in benthic habitat mapping with only 10% of available labels - and strong cross-domain generalization, outperforming fully supervised baselines on the AI4Shipwrecks benchmark using frozen representations.

0.55

Catching the Imposter: Self-Supervised Learning of Physical Coherence with Cross-Entity Feature Permutations

cs.LG ·2026-08-14 ·Aleksei Rozanov, Arvind Renganathan, Vipin Kumar
similarity 0.57
Abstract

Scientific data often describe entities whose features are jointly governed by the laws of physics, yet existing self-supervised learning (SSL) objectives largely ignore this physical coherence. We introduce imposter, a discriminative pretext task that replaces subsets of an entity's features with real observations donated by another entity and trains the encoder to identify the swapped features. Because every donated value is individually plausible, the task can only be solved by learning cross-feature physical dependencies. We evaluate the proposed objectives on global ERA5-Land reanalysis data using 21 environmental variables and assess the learned representations on seven downstream tasks spanning climate classification, carbon flux estimation, and streamflow prediction. Our study includes, to our knowledge, the first systematic comparison of self-supervised objectives for land-surface modeling under a shared architecture and pre-training budget. We find that the most effective pretext task depends on the downstream task family rather than any single objective's superiority, and that imposter provides complementary information when combined with existing SSL objectives. These results suggest that physical coherence is a valuable new source of self-supervision for scientific foundation models.

0.04