CS Paper Radar

Are Seismic Foundation Models More Than Glorified Feature Extractors?

Laura Laurenti, David Liu, Brian Kulis, Men-Andrin Meier, Christopher Johnson

cs.LG cs.AI ·submitted 2026-10-27 21:54 UTC · v1
position

Abstract

Earthquake sequences are stochastic on two separated timescales: the seismic waves of any individual earthquake propagate over seconds to minutes, while the earthquakes themselves arrive as a sparse, self-exciting occurrence process, with inter-event intervals from seconds to months. Respecting both scales in one training objective is the open challenge. Foundation models (FMs) for seismic waveforms instead import their learning targets from audio and vision, where every target is computed from a single fixed-length window. Such a window holds one realization of the wave propagation (an earthquake trace), never the occurrence process that decides when earthquakes happen. We develop nine model designs sharing one Mamba encoder-decoder, trained on a fixed corpus of 3M windows, varying the self-supervised objective, quantization bottleneck and mixer; the two key comparisons are architecture-matched, varying the objective alone. Each checkpoint is evaluated frozen on three downstream tasks. Varying the target redistributes downstream skill rather than raising it, and no checkpoint wins everywhere. Adding masking pressure to our strongest classifier re-specializes it rather than making it a generalist: it gains 6.9 points of few-shot P-wave picking precision and loses 9.6 points of few-shot classification accuracy. By the standard imported from language and vision, these models are feature extractors rather than FMs. Controls on capacity, architecture, and domain proximity do not explain why skill moves rather than rises. Against task-specific baselines the advantage is regime-dependent: our checkpoints beat them few-shot on classification and ground-motion prediction, and trail them on full data. We argue the binding constraint is the target, and that settling the question requires a learning target defined across events, the timescale that makes seismicity stochastic in the first place.

Comment
NEURIPS STODY WORKSHOP 2026

Closest papers

Nearest neighbours by meaning across the whole index.

SeisMamba: Low-Latency Single-Station Seismic Magnitude Estimation for Spatially Distributed Earthquake Early Warning

cs.LG ·2026-08-25 ·Quenton Yeo, Zhaoge Bi, Linghan Huang, Luke Stephen Higgins, Flora Salim +1 more
benchmark similarity 0.68
Abstract

Rapid earthquake magnitude estimation is central to earthquake early warning, yet many operational systems depend on dense regional seismic networks and region-specific calibration. This creates a spatial coverage barrier for high-risk areas with sparse sensing infrastructure. Single-station learning offers a lower-cost alternative, but existing models often face an accuracy--latency trade-off and may degrade under regional distribution shift. We present SeisMamba, a lightweight Mamba-based architecture for low-latency magnitude estimation from minimally processed three-component seismic waveforms recorded at a single station. SeisMamba combines hierarchical convolutional encoding, sparse selective state-space modelling, multi-scale feature fusion, and an auxiliary temporal prediction head to support efficient long-sequence waveform analysis. On the STEAD benchmark, SeisMamba achieves the best MSE, RMSE, and $R^2$ among tested baselines while requiring only 0.55 ms for a batch of 32 waveforms on an NVIDIA T4 GPU, making it about three times faster than transformer-based baselines. We further conduct a Chile--Taiwan regional hold-out experiment as a diagnostic test of cross-region deployment, where SeisMamba retains useful performance on geographically unseen seismic regions. These results suggest that selective state-space waveform modelling provides a promising accuracy--latency backbone for spatially distributed, low-cost earthquake early warning.

0.17

A Seismic Foundation Model for Dense Feature Extraction

cs.CV ·2026-11-16 ·Zhixiang Guo, Xinming Wu, Yimin Dou, Hui Gao, Siyao Zhao +2 more
similarity 0.67
Abstract

Deep-learning-based seismic interpretation is constrained by scarce high-quality annotations, and most existing models are trained separately for individual tasks. We present SeisDFE, a 7B-parameter seismic foundation model for dense feature extraction, pretrained on 5.34 million 2D images derived from more than 500 seismic volumes. A three-stage pretraining strategy, combining discriminative self-supervised learning, Gram feature anchoring, and multi-resolution adaptation, improves feature discriminability while preserving spatial coherence. With its encoder frozen, SeisDFE supports seismic facies classification, salt detection, fault detection, and relative geologic time estimation. It outperforms existing seismic foundation models on all four tasks and a natural-image-pretrained model on all but fault detection, with the largest gains under scarce supervision: using only two annotated sections of the F3 dataset, it improves whole-volume mIoU by 12-19 percentage points over the baselines. The learned features remain effective with task-specific decoders and extend to synthetic-to-field transfer and sparse 3D geobody interpretation. These results show that large-scale seismic pretraining organizes geological information into dense representations that are directly reusable across interpretation tasks.

0.18

Physical Cross-Modal Masked Autoencoding for Seismic-to-Well Representation Learning

cs.LG physics.geo-ph ·2026-09-28 ·Meher Gajula, Keyla Gonzalez, Ben Lasscock, Alejandro Valenciano
similarity 0.62
Abstract · 11 pages, 3 figures

Learning from scientific measurements often requires aligning modalities with different spatial support and resolution. Subsurface characterization is an extreme dense-sparse problem in which 3D seismic provides volumetric but indirect measurements and well logs provide high-resolution 1D measurements at sparse borehole locations. We introduce a physically grounded cross-modal masked autoencoder (CM-MAE) for seismic-to-well representation learning. The model jointly tokenizes seismic volumes and well-log depth patches, embeds both modalities in continuous physical coordinates using four-axis rotary position embeddings, and reconstructs masked targets with a cross-modal decoder. Sparse Mixture-of-Experts layers provide modality-specific capacity while dense attention allows information exchange between modalities. Pretraining uses 23 seismic volumes covering approximately 178,000 square km across U.S. offshore and onshore basins, together with approximately 92,000 wells. Matched-mask ablations show strongly asymmetric information flow. Seismic context improves held-out well-log reconstruction by 9.73%, while well-log context improves seismic reconstruction by 0.65%. We evaluate seismic-only pseudo-log predictions against independent interpreter-drawn salt-geobody masks to determine whether they contain geologic signal. Across offshore U.S. surveys, compressional-slowness-derived salt scores reach an AUROC of 0.910. In onshore basins, predicted compressional slowness preserves formation-scale structure and partial relative organization in an unseen survey despite calibration drift. CM-MAE learns useful seismic-to-well representations under extreme modality asymmetry, although absolute pseudo-log calibration remains survey-dependent.

0.21

From Idle to Urgent: A Resource-Harvested HPC Workflow for High-Fidelity Seismic Estimation

cs.DC cs.CE ·2026-09-19 ·Tsuyoshi Ichimura, Kohei Fujita, Hideaki Ito, Wataru Sakurai, Muneo Hori +1 more
similarity 0.60
Abstract · 26 pages, 4 figures, accepted for Eighth Workshop on Interactive and Urgent High-Performance Computing

We propose an Urgent Interactive HPC workflow that dynamically integrates high-fidelity 3D nonlinear analysis with surrogate neural networks (NNs) to enable rapid decision-making during large-scale earthquakes. This approach achieves both "Resource Harvesting", which utilizes idle computing capacity during non-emergency periods, and immediate response during crises. By developing two specialized HPC kernels, the proposed method reduces energy-to-solution by 76% and improves throughput by 3.7-fold during normal operations to efficiently construct training datasets, while during emergencies, it couples NN-based inverse analysis with physics-based simulations and dynamic refinement to reduce conventional computational costs by over 97.9%, enabling the generation of highly reliable spatial time-history ground motion distributions within 30 minutes post-earthquake.

0.38

Event-Centric Earthquake Monitoring with a Multi-Station Transformer

cs.AI physics.geo-ph ·2026-10-12 ·Haoyu Wang, Weiqiang Zhu
similarity 0.59
Abstract · 17 pages and 7 figures in main body; 16 figures and 4 tables in supplementary

Earthquake catalogs underpin studies such as fault structure, earthquake processes and seismic hazard, and their value depends on being both complete and reliable. Machine learning has improved individual tasks of earthquake monitoring, but processing stations independently discards the consistency of arrivals across the network, and running tasks in sequence fragments the information describing each earthquake. We present EQNet2, a geometry-aware multi-station Transformer that jointly predicts phase arrivals, first-motion polarities, source location and magnitude, and the association of arrivals into events. On a Northern California test dataset and on continuous waveforms from Ridgecrest and the San Francisco Bay Area, EQNet2 recovers more weak arrivals, produces fewer false detections, and associates more picks into located events. Its learned attention links information across stations along time--distance patterns consistent with seismic moveout. As seismic networks grow denser, exploiting multi-station coherence and event-centric prediction offers a path to more complete and reliable catalogs.

0.22

Breaking feature collapse in self-supervised time series encoders

cs.LG cs.AI ·2026-11-16 ·Özgün Turgut, Daniel Rueckert
similarity 0.58
Abstract

General-purpose time series encoders promise a single backbone that generalises across tasks and domains; however, they currently do not scale. We trace this down to a critical point of failure: under the predominant masked data modelling supervision, the encoder's output tokens partially collapse. This is the natural consequence of supervising encoders only in the input space, which leaves no constraint against redundant token dimensions in the feature space. We introduce OTISv2, which suppresses the collapse by complementing the reconstruction objective in input space with two objectives in the feature space: a self-distillation loss against an exponential-moving-average teacher, and a KoLeo regulariser. Under matched training, the resulting encoder reduces the cosine similarity and lifts the effective rank of its output tokens relative to its predecessor, and it scales monotonically from 7.1 M to 116 M parameters where its predecessor declines. It matches or surpasses existing general-purpose time series encoders across 143 uni- and multi-variate datasets. To foster further research, we release our code and pre-trained weights.

0.27

An AI-assisted conditioning and geological interpretation workflow for usage in implicit geological modeling

cs.AI physics.geo-phcs.LG ·2026-10-07 ·Stefan Carpentier, Jan Diederik van Wees, Eva de Boever, Jan Niederau, Camille Chapeland +3 more
similarity 0.58
Abstract · 17 page, 19 figures

Implicit modeling and Relative Geologic Time are geological modeling techniques that enable more efficient, faster, less biased and more reproducible modeling results. For optimal operation, these techniques require many well-constrained input data. In the framework of the Horizon Europe GO-Forward and MOOI WarmingUP GOO projects and to accelerate Implicit modeling, Machine Learning (ML) methods have been tested and implemented in a toolkit for the interpretation of (onshore) seismic data from the shallow to deep range (+- 300 - 3500 m). The goal is to rapidly characterise this depth domain by efficient interpretation of horizons and faults in seismic data. The first step is to improve the signal by applying AI techniques like self-supervised and semi-supervised contrastive learning CNN's for noise reduction and interpolation. Next, horizons and faults are interpreted with minimal use of human-generated training data by using (semi-) self-supervised methods. The resulting developed toolkit supports the application of the implemented algorithms in an efficient workflow. As a first demonstration, the top of the Dutch Maassluis Formation has been interpreted in the Leeuwarden and Waalwijk 3D seismic cubes. Overall, this study demonstrates that AI-assisted interpretation workflows have reached a level of maturity that allows their integration into applied geological modeling and decision-making.

0.22

Should All Noises Be Treated Equally: Impact of Input Noise Variability on Neural Network Robustness

cs.LG eess.SPphysics.geo-ph ·2026-09-13 ·Salma Alsinan, Maksim Makarenko, Sixiu Liu, Ali Aldawood, Ibrahim Hoteit
published similarity 0.58
Abstract · Main:(pages 1-31,15 figures), Supplementary: (pages: 32-75, 37 figures)

Geophysical data collected from active field sites are often contaminated by complex and heterogeneous noise, obscuring weak seismic events, and complicating automated interpretation. Although deep learning offers promising solutions for seismic processing, its performance is highly sensitive to the nature of training noise, especially under out-of-distribution (OOD) conditions. This study investigates the influence of noise parameters, such as type, scale, and complexity on the performance, generalization and robustness of neural networks in two geophysical tasks: first break picking and denoising. We simulate seismic-while-drilling data and apply controlled input source noise augmentation using stochastic generators to vary the noise characteristics. Different neural networks are trained on fixed noise types and scales, then evaluated across both seen and unseen noise scenarios. We incrementally increase the complexity of the noise by introducing compound noise mixtures and assess the performance of the model under increasingly challenging OOD conditions. This yields a robustness matrix that captures the generalizability of each model relative to its training configuration. Results indicate that larger noise scales boost generalization, and that effective alignment between noise type, task complexity, and architecture is key for maximizing generalization gains. In addition, training with compound noises mitigate weaknesses associated with single-noise training, acting as an additional implicit regularizer to improving robustness. These findings highlight key factors influencing model resilience in noisy geophysical environments and offer guidance for developing deep learning models that generalize effectively across diverse and unpredictable noise conditions.

0.49