CS Paper Radar

Mesh-Native Tokenisation: Retrofitting Foundation Models to Unstructured Data

Georg Maierhofer, Miles Cranmer, Payel Mukhopadhyay

cs.CE cs.AImath.NA ·submitted 2026-10-08 18:37 UTC · v1

Abstract

Physics foundation models are an increasingly popular approach for building versatile emulators for spatiotemporal systems. They require tokenisation layers that lift physical field data into a latent representation suitable for dynamic processing. A typical choice for this layer is a convolutional encoder, which lifts field information in a locality-aware way and can be trained very efficiently. However, this choice cannot be applied to a large class of scientific computing problems where data are represented on unstructured meshes, for example simulations on domains with curved boundaries. Such data typically require graph neural network architectures, which are more expensive to train and, in the form currently used, encode information differently from their CNN counterparts, thus limiting the portability of physics emulators between these data regimes. In this work we introduce Mesh-Native Convolution (MNC), a locality-aware tokenisation layer that encodes unstructured mesh-based data in a way that is analogous to CNNs. This approach leads to models that can be trained once on mesh data and then deployed to arbitrary unstructured meshes, even if these are irregular, with minimal retraining. We show that this tokenisation approach can be used to retrofit grid-pretrained foundation models efficiently on unstructured meshes and provide theoretical results to confirm the consistency of this approach with classical CNN encoders. We demonstrate the favourable performance of this retrofitting approach, which in many cases reaches acceptable accuracy even in zero-shot deployment, and outperforms existing baselines after only a few epochs of fine-tuning, thus providing an efficient bridge between the two data regimes.

Closest papers

Nearest neighbours by meaning across the whole index.

HI-MeshGraphNets: Efficient and Accurate Mesh-based Physics Learning with Hierarchical Multi-scale Graph Neural Networks

cs.LG ·2026-08-13 ·SiHun Lee, Dong-Hyuk Park, Taesoo Bang, Seung-Hoon Kang
similarity 0.70
Abstract

Machine-learned physical surrogate models have become promising alternatives to mesh-based numerical solvers. Among them, graph neural networks (GNNs) are well suited for representing simulation meshes and learning nodal state evolution through message passing. However, conventional flat message passing becomes inefficient on large, high-fidelity meshes because information propagates only one hop per layer, requiring deep processors for long-range interactions and increasing computational cost, memory usage, and the risk of over-smoothing. To address this limitation, we propose Hierarchical Interpolating MeshGraphNets (HI-MGN), a multiscale extension of MeshGraphNets for efficient long-range communication on unstructured meshes. HI-MGN replaces the flat processor with a hierarchical multiscale processor that coarsens graphs using farthest-point sampling and Voronoi partitioning while preserving the original mesh topology. Message passing on coarse graphs enables information to travel over larger geometric distances with fewer layers, and a learned graph interpolation network reconstructs fine-resolution features. Across three structural and fluid benchmarks, HI-MGN achieves improved accuracy compared with MeshGraphNets and the Bi-Stride Multi-Scale GNN while reducing training time and peak memory usage. The results show that topology-aware hierarchical message passing and learned coarse-to-fine interpolation provide an effective and practical framework for scalable mesh-based physics surrogate modeling.

0.04

Geometry Meets Physics: Data-Efficient Pre-Training for Unstructured Neural PDE Solvers

cs.AI ·2026-10-02 ·Luis Medrano-Navarro, Giacomo Baldan, Qiang Liu, Benjamin Holzschuh, Jan Hagnberger +2 more
top venuecode/data similarity 0.65
Abstract · Accepted to NeurIPS 2026. Code and models are available at https://github.com/tum-pbs/GMP

Neural surrogate models for Partial Differential Equations (PDEs) on unstructured 3D geometries are often limited by poor generalization and the high cost of generating large-scale training datasets. Consequently, pre-training on massive datasets of related PDE dynamics has emerged as a critical alternative to enhance the robustness and scalability of these models. However, this strategy is neither compute- nor data-efficient, as it relies on massive pre-computed data that is very costly to generate. In this work, we introduce a disk-data-free pre-training framework tailored to both steady-state and transient regimes. For steady-state problems, we propose a geometry-driven strategy that leverages intrinsic shape descriptors to learn representations of complex 3D domains. For transient problems, we introduce a physics-driven approach based on online generation of synthetic PDE data, enabling scalable pre-training without reliance on expensive datasets. Across multiple experiments, our approach achieves faster convergence, greater data efficiency, and higher accuracy during fine-tuning, particularly under realistic low-data regimes. This methodology provides a practical pathway toward data-efficient neural emulators for large-scale simulations. Code and models are available at https://github.com/tum-pbs/GMP.

0.43

OceanLight: Efficient Global Ocean Forecasting via Geometry-Adaptive Unstructured Mesh Representation

cs.LG cs.AI ·2026-08-17 ·Wei Wu, Xiang Wang, Hongze Leng, Qingye Min, Junxing Zhu +1 more
similarity 0.64
Abstract · 35 pages, 21 figures

Reliable global ocean forecasting is critical for climate monitoring, marine navigation, and extreme event early warning. Physics-based ocean forecasting models impose prohibitive computational costs, while existing deep learning approaches predominantly rely on structured-grid architectures, incurring unnecessary computation on masked land cells and enforcing uniform resolution across dynamically heterogeneous ocean regions regardless of local flow complexity. Here we present OceanLight, an efficient global ocean forecasting framework innovatively combining geometry-adaptive unstructured mesh tokenization with a graph neural network (GNN) backbone. OceanLight achieves pointwise forecast accuracy and kinetic energy spectral fidelity exceeding both operational numerical analyses and state-of-the-art AI-based models, while surpassing all AI-based ocean models in geostrophic balance consistency. Furthermore, OceanLight demonstrates reliable mesoscale eddy representation, capturing coherent ocean structures beyond pointwise statistical optimization. These capabilities are delivered with a 62% reduction in GPU memory consumption and 70\% reduction in FLOPs relative to structured-grid baselines. Our unstructured mesh representation establishes a generalizable paradigm for scalable data-driven oceanography.

0.29

Application of PDE emulators for a multi-material, multi-physics simulation problem

cs.LG ·2026-10-27 ·Diane Oyen, Morgan Jones, Adithan Kathirgamaraju, Abu Bucker Siddik, Alexander Most +17 more
published similarity 0.63
Abstract · Accepted at ICMLA 2026, Rochester, MI, USA

Machine learning (ML) architectures are being developed for fast emulation of numerical simulators to accelerate both scientific discovery and engineering design. While these approaches are well-tested on popular PDE (partial differential equation) benchmarks, it is necessary to evaluate their performance on datasets that more accurately reflect industrial and engineering applications. We evaluate several computer vision-based architectures trained as surrogate models on a challenging computational physics task involving multiple materials, revealing significant implementation challenges for adapting certain pre-trained models, including transformer-based architectures. For such multi-physics multi-material datasets, our results underscore the importance of learning algorithms designed for sparse and imbalanced data, techniques for adapting spatial encodings and physical consistency metrics, in addition to methods for reducing the cost of generating sufficient training data.

0.25

Mesh2Shell Neural Operator for Inverse Shape Optimization

cs.LG ·2026-11-01 ·Xinyi Li, Xi Deng, Yubo Cao, Hudson Liu, Anima Anandkumar
similarity 0.62
Abstract

Physics-driven design seeks high-performing geometries, which need not conform to a learned shape category. Parametric and latent-space approaches facilitate surrogate-based optimization, but restrict deformations to prescribed shape families or learned decoders. We introduce the Mesh-to-Shell Neural Operator (M2S-NO) as the mesh-to-shell component of a category-agnostic inverse shape optimization framework. We separate the explicit mesh used for design from the volumetric representation used for physics prediction. M2S-NO maps oriented surface samples to a surface-centered shell representation, such as TruncatedSDF or HeatSDF. A separate problem-specific PDE neural operator (PDE-NO) maps this representation to physical fields. Objective gradients update mesh vertices through both operators, without a latent shape decoder. Preconditioning and remeshing support free-form deformation, while fixed interfaces are controlled directly on the mesh. Optional shell-space proposals enable topology changes. We evaluate geometric matching, jet-engine brackets, and neutron moderators, with independent FEM and OpenMC evaluation. Relative to the best training designs, M2S-NO reduces maximum bracket displacement by 14.3% and increases neutron-sensor current by 13.8%. Optimization also succeeds from an out-of-distribution bracket initialization, demonstrating the framework's geometric flexibility.

0.26

SimpFlow: Transferable Pretraining for Fluid Dynamics via Simplified Physics

cs.CE ·2026-11-09 ·Ruijie Li, Tianbai Xiao
benchmark similarity 0.62
Abstract · 29 pages, 20 figures, 19 tables

Neural surrogates enable rapid fluid simulation, but learning across diverse three-dimensional (3D) geometries requires costly computational fluid dynamics (CFD) labels. We introduce SimpFlow, a pretraining approach that generates supervision from simplified physics on unlabeled geometry libraries. A source-panel potential-flow solver generates velocity and pressure fields for approximately 9,000 3D geometries, capturing geometry-induced acceleration, stagnation, and suction while omitting viscosity and separated wakes. The solver imports only a surface mesh and reuses precomputed velocity fields across inflow directions, enabling inexpensive label generation. In our benchmark, amortized labeling takes 0.111 s per example using four CPU threads. We pretrain a neural surrogate on these labels and fine-tune it on independent CFD datasets. Across five flow tasks, SimpFlow improves over training from scratch in all 24 task-budget settings. With fine-tuning fixed at 200 epochs, interpolation indicates that SimpFlow requires 35-68% fewer CFD training cases to match the error of scratch models trained at each task's largest data budget. These results demonstrate that simplified physical solutions can provide transferable supervision, offering a practical way to reduce the high-fidelity data requirements of fluid surrogates.

0.23

TokenFlow: Latent Token Rectified Flow Matching for 3D Point Cloud Generation

cs.CV ·2026-10-20 ·Linhao Wang, Dongjin Huang
similarity 0.61
Abstract

3D point clouds generation requires capturing both part-level layout and surface detail while preserving permutation equivariance. Existing latent generative models often encode an entire shape into a single vector, which limits their ability to retain part-level structure. We propose TokenFlow, a hierarchical flow matching framework that represents each shape as a set of latent tokens and couples a rectified flow over these tokens with a conditional flow over points. A bottleneck of $M \ll N$ learnable tokens aggregates global context through cross-attention at $O(NM)$ cost. On ShapeNet, TokenFlow achieves the best MMD and coverage across airplane, car, and chair, while obtaining 1-NNA comparable to the strongest baselines. On ModelNet, its learned latent set reaches the best reported linear-evaluation accuracy on ModelNet10. These results show that representing a shape as a set of latent tokens, aggregated at $O(NM)$ cost, provides an effective representation for 3D point cloud generation.

0.07

Beyond Pairwise Graphs in Science: Hypergraph Adaptive Wavelet Operators for Parametric PDEs

cs.LG physics.comp-ph ·2026-08-28 ·Rajat Sarkar, Venkataramana Runkana, Souvik Chakraborty
similarity 0.61
Abstract

Physical systems are often modeled by solution operators that map input fields, parameters, geometries, or past states to steady or future physical states. Learning these maps is difficult, especially for time-dependent systems that must assimilate history and remain stable under autoregressive rollout. Many neural operators work best on regular, structured grids, while realistic simulations often require unstructured meshes or point clouds to resolve complex geometries; in such settings, grid-centric representations can lose accuracy. Graph neural operators handle these domains through message passing or spectral graph filtering, but pairwise edges do not directly capture group-wise couplings among mesh cells, local neighborhoods, or conservation volumes. We introduce the Hypergraph Adaptive waveLet Operator (HALO), which lifts the domain to a hypergraph and learns in its spectral wavelet domain. HALO avoids explicit hypergraph-Laplacian eigendecomposition through Chebyshev polynomial wavelet filters, giving localized spectral kernels at linear sparse-matrix cost. Its trainable dyadic wavelet scales are regularized toward tight-frame coverage, allowing the frequency response to adapt to each PDE while encouraging stable multi-scale spectral coverage. Across 2D and 3D benchmarks on structured and unstructured discretizations, HALO achieves best or near-best accuracy among frequency-, transformer-, DeepONet-, state-space-, and graph-based baselines and sustains stable multi-step rollouts. The same model scales to industrial aerodynamic geometries: on meshes of a few hundred thousand points it is on par with, or better than, the strongest fixed-discretization transformers, while remaining resolution-equivariant.

0.19