CS Paper Radar

ISLA: A Coordinate-Free, Measure-Weighted Transformer for Boundary-Driven PDEs

Peter Sharpe

cs.LG cs.CEmath-phphysics.flu-dyn ·submitted 2026-11-12 17:30 UTC · v2

Abstract

Symmetry is a foundational principle of physics: the behavior of a PDE-governed system does not depend on the coordinate frame, unit system, or mesh used to write it down, and neither should a surrogate model. Yet today's leading neural surrogates lack these invariances and fail under simple transformations of the same problem: for models trained on PDEs of cars in one pose (i.e., translation and rotation), rotating the full PDE raises the surface-pressure error from 4% to more than 100%, and re-sampling its boundary raises it to about 30%. We introduce ISLA (Invariant Slice Attention), a slice-attention transformer for steady, boundary-driven PDEs. Because the boundary determines the interior solution, ISLA encodes the posed boundary once into learned slices and evaluates arbitrary interior points independently through a tree-accelerated learned boundary kernel. Inputs are nondimensionalized in scales computed from the boundary, sums over boundary cells are area-weighted quadratures that converge under mesh refinement, and geometry enters only as distances and relative angles to equivariant slice anchors, making the model equivariant to rigid motions and reflections up to optional token subsampling and tree clustering. On AeroZoo, 2,816 3D flows spanning diverse geometries, poses, physical scales, and boundary conditions, ISLA's mean reference-scaled velocity and pressure errors are at least 1.9 and 2.2 times lower than any baseline's. On full AirfRANS, ISLA is second only to an explicit boundary-integral network (GLOBE), with errors at least 1.6 times lower than Transolver's and GeoTransolver's in every field.

Comment
49 pages, 14 figures, 27 tables. Companion paper to "AeroZoo: A Benchmark of Well-Posed Flow Problems for Foundation Models of Aerodynamics" (Sharpe et al., 2026, arXiv:2611.17152)

Closest papers

Nearest neighbours by meaning across the whole index.

HiLNO: A Hierarchical Latent Neural Operator with Multi-Scale Supervision for PDEs on General Geometries

cs.LG math.NA ·2026-09-16 ·Zhicheng Hu, Jiacheng Li, Min Yang
code/data similarity 0.66
Abstract

Latent neural operators improve the efficiency of operator learning for partial differential equations (PDEs) by performing the main computation on compact latent representations. However, directly compressing the input representation to obtain such compact representations may discard solution-relevant spatial information, especially for PDE solutions with multiscale structures. To address this problem, we propose HiLNO, a hierarchical latent neural operator that constructs a fine-to-coarse-to-fine latent space and further introduces multi-scale supervision (MSS) and anisotropic Gaussian attention. The hierarchy mitigates potential information loss during compression, while MSS aligns intermediate predictions with downsampled target fields, encouraging solution-relevant structures to be captured across multiple spatial scales. Anisotropic Gaussian attention enables feature transfer across the hierarchy, making HiLNO applicable to general geometries. Experiments on representative PDE benchmarks and a large-scale automotive aerodynamics task show that HiLNO achieves competitive predictive accuracy, while reducing the parameter count by an average of 84.4% and FLOPs by an average of 69.2% compared with LinearNO. Additional experiments demonstrate effective generalization to unseen spatial resolutions. Code is available at https://github.com/JcLimath/HiLNO.

0.25

BAT-NO: A Boundary-Condition-Aware Transformer Neural Operator for Crashworthiness Prediction of Vehicle Components

cs.LG ·2026-10-02 ·Haoran Li, Yingxue Zhao, Haosu Zhou, Mustapha Ziane, Pierre Culiere +2 more
similarity 0.66
Abstract

High-fidelity finite-element simulations provide accurate crashworthiness predictions, but their cost limits iterative design exploration. Deep learning surrogates can reduce this cost, but many component-level models are developed under a single prescribed boundary condition, limiting generalisation to boundary variations. This work proposes a Boundary-Condition-Aware Transformer Neural Operator (BAT-NO) for autoregressive prediction of transient displacement fields and scalar crashworthiness responses under variations in geometry and boundary conditions. A B-pillar simulation framework evaluates generalisation across variations in geometry, impact position and velocity, and support stiffness. BAT-NO combines recurrent mesh processing with latent-grid Fourier operator processing. Boundary-condition information is transferred to the latent grid through a hybrid local--global mechanism. Slice-based attention models interactions among physically related regions, while direct boundary-to-grid projection preserves local spatial structure. Across the validation sets for the shape-only, shape-and-loading, and shape-loading-boundary cases, BAT-NO achieves the lowest mean final-step mean nodal Euclidean displacement error among the evaluated baselines. In the most challenging case, it reduces the mean error by 32.6% relative to the second-best model. Hyperparameter tuning reduces the validation error from 0.451 to 0.269 mm, with a comparable error of 0.267 mm on 300 unseen test simulations sampled within the investigated design space. An attention-based scalar decoder jointly predicts six response trajectories with a mean relative error of 2.46%. Most derived crashworthiness indicators have median errors below 3%. These results show that explicit local and global boundary-condition representations improve crashworthiness prediction over expanded component-level design spaces.

0.21

SCANO: Restoring Boundary-Condition Completeness in Sampled Meshes with a Sliced-Cross-Attention Neural Operator for 3D Thermal Fields

cs.LG physics.comp-ph ·2026-11-03 ·Tianyu Li, Zhiwei Cao, Qingang Zhang, Ruihang Wang, Binyang Song +1 more
benchmark similarity 0.65
Abstract · 88 pages (46 main text + 42 supplementary material)

Training neural operators on million-cell meshes can exceed GPU memory limits. Models are therefore trained on subsamples and deployed on whole meshes or selected regions. However, subsampling can distort boundary information, degrading accuracy. We ask when boundary conditioning under sampling stays consistent with whole-mesh training, and how to aggregate and inject boundary data. Representing the solution as an integral of boundary data against type-dependent measures, we show that subsampling distorts this conditioning in three ways: incomplete boundary coverage or a sample-dependent global state, face weighting by count rather than area, and compression of boundary inputs into a low-dimensional vector. This analysis yields four measurable consistency conditions checkable before training. We satisfy them with two backbone-agnostic modules. Boundary Quadrature Attention (BQA) aggregates all boundary faces, typed and area-weighted, into slice tokens that every query point reads at every layer. Anchored Global State (AGS) estimates the global state from anchors independent of the sampled points. Adding both to a scalable message-passing and slice-attention backbone yields SCANO. We evaluate the approach on controlled and public benchmarks and a data-centre thermal simulation dataset (DCTS) containing million-cell meshes. On both benchmarks, BQA improves every backbone, reducing whole-mesh error up to 15-fold. On all three datasets, AGS aligns regional with whole-mesh predictions, cutting regional error at least tenfold for most backbones on the public benchmark. Together, the modules nearly halve the best original backbone's whole-mesh error on both benchmarks. SCANO predicts a room-scale field in under one second, over 100 times faster than whole-room anchor self-attention.

0.33

CIS-Transolver: A Cross-layer Incremental-State Neural Operator for Fine-Scale Automotive Aerodynamic Flow Prediction

cs.LG cs.AI ·2026-10-19 ·Jianglong Liu, Zhicheng Zhu, Jia Hao, Liangyue Jia, Wenxuan He
similarity 0.65
Abstract

Neural operators offer an efficient data-driven approach to predicting complex automotive aerodynamic flows. However, spectral bias limits their ability to reconstruct fine-scale flow variations around geometric features such as side mirrors and wheel arches. Reliable assessment of local geometric changes therefore requires models that capture fine spatial detail together with global flow interactions. Inspired by iterative refinement and state evolution, we propose CIS-Transolver, a cross-layer incremental-state neural operator that integrates local multiscale information into global flow modeling. The method extracts local feature variations and encodes them as zeroth- to second-order geometric moments. These encodings are combined with information from earlier layers to form a cross-layer memory. Memory-derived corrections enter the value branch of Physics-Attention, allowing local details to inform global physical-state interactions throughout the network. We systematically evaluate CIS-Transolver on three public automotive aerodynamics benchmarks, ShapeNet-Car, AhmedML, and DrivAerNet++, covering surface pressure fields and volumetric pressure and velocity fields. CIS-Transolver achieves the lowest mean relative L2 error among the evaluated methods for all five prediction metrics across the three benchmarks. Relative error reductions average 8.72% compared with the best baseline for each metric. Also achieves the lowest high-frequency spectral errors across all five metrics, with an average relative reduction of 16.06% compared with the best baseline. These results support cross-layer reuse of local information for fine-scale automotive flow prediction.

0.21

Cluster Attention Neural Operators for Solving Parametric Partial Differential Equations

cs.AI math-phcs.LGmath.DS ·2026-09-30 ·Ming Zhong, Antonio Colanera, Gianluigi Rozza, Zhenya Yan
similarity 0.64
Abstract · 30 pages, 9 figures

Traditional simulations of parametric partial differential equations (PDEs) rely on repetitive computations for each parameter, which makes high-fidelity design impractical. Neural operators address this issue by learning solution operators, accelerating parameter-space mapping by orders of magnitude. Recent Transformer-based neural operators attempt to capture global dependencies, but often at the cost of quadratic attention complexity. Transolver resolves this problem by projecting physical states into a reduced slice space for attention computation. Although fast, this projection sacrifices fine spatial information. Moreover, by operating in this reduced space with shared weights across attention heads, it may constrain the model's flexibility, thereby limiting its capacity to capture complex phenomena. To address these issues, we propose the Cluster Attention Neural Operator (CANO), which reformulates attention via a novel cross-attention mechanism that dynamically clusters queries while preserving full-resolution keys and values. This avoids slice compression loss and removes weight-sharing limits. At the same time, the model remains fast without losing global interactions. Empirically, CANO achieves state-of-the-art performance across canonical PDE benchmarks, covering fluid and solid dynamics (e.g., Navier-Stokes, Airfoil, Plasticity), irregular unstructured geometries (e.g., Pipe Turbulence, Composites), and long-term temporal rollouts. Across solid deformation and turbulent flow benchmarks, CANO achieves lower errors than baselines and exhibits strong geometric adaptability and temporal consistency.

0.26

Geometry Meets Physics: Data-Efficient Pre-Training for Unstructured Neural PDE Solvers

cs.AI ·2026-10-02 ·Luis Medrano-Navarro, Giacomo Baldan, Qiang Liu, Benjamin Holzschuh, Jan Hagnberger +2 more
top venuecode/data similarity 0.64
Abstract · Accepted to NeurIPS 2026. Code and models are available at https://github.com/tum-pbs/GMP

Neural surrogate models for Partial Differential Equations (PDEs) on unstructured 3D geometries are often limited by poor generalization and the high cost of generating large-scale training datasets. Consequently, pre-training on massive datasets of related PDE dynamics has emerged as a critical alternative to enhance the robustness and scalability of these models. However, this strategy is neither compute- nor data-efficient, as it relies on massive pre-computed data that is very costly to generate. In this work, we introduce a disk-data-free pre-training framework tailored to both steady-state and transient regimes. For steady-state problems, we propose a geometry-driven strategy that leverages intrinsic shape descriptors to learn representations of complex 3D domains. For transient problems, we introduce a physics-driven approach based on online generation of synthetic PDE data, enabling scalable pre-training without reliance on expensive datasets. Across multiple experiments, our approach achieves faster convergence, greater data efficiency, and higher accuracy during fine-tuning, particularly under realistic low-data regimes. This methodology provides a practical pathway toward data-efficient neural emulators for large-scale simulations. Code and models are available at https://github.com/tum-pbs/GMP.

0.65

FSAN: Flow State Attention Network for Aerodynamic Prediction

cs.CV cs.AI ·2026-09-06 ·Wenxuan Jin, Jianguo Yao, Haibing Guan, Xijun Li
similarity 0.63
Abstract

Accurate aerodynamic prediction is critical for designing fuel-efficient and safe transportation systems such as aircraft and automobiles, yet traditional computational fluid dynamics (CFD) simulations remain computationally expensive and expertise-intensive, severely limiting their use in iterative design and real-time analysis. Existing deep learning surrogates suffer from two major limitations: (i) they are evaluated on datasets with narrow flow-condition ranges, leaving their performance under complex flow conditions undemonstrated; (ii) they treat global flow conditions as a single vector injected uniformly across all surface points, ignoring that different geometric regions experience distinct local flow phenomena, which degrades prediction accuracy under complex flow conditions. To address these limitations, we propose the Flow State Attention Network (FSAN). FSAN separately encodes point cloud and flow conditions, then partitions the geometry into multiple flow states via learnable soft assignments, and uses flow features to update these state representations, which in turn influence point cloud features through state changes. This enables fine-grained, state-specific interaction between geometry and flow information. Extensive experiments on two well-recognized aerodynamic benchmarks demonstrate that FSAN achieves the highest accuracy among the methods compared in this work at a higher computational cost. On Emmi-Wing, FSAN reduces the Relative L2 (REL-L2) error by over 20\% compared to the strongest baseline (Transolver), and on DrivAerNet++, it achieves a 10\% reduction compared to the strongest baseline (AdaField). These results establish FSAN as a promising neural surrogate on public benchmarks with diverse flow conditions and geometries.

0.22

Dynamic Quadtree Tokenization and Transformer for Adaptive Mesh PDE Forecasting

cs.CV cs.LG ·2026-10-02 ·Yilin Zhuang, Noah Zambrano, Karthik Duraisamy
code/data similarity 0.62
Abstract

The quadratic attention cost of Vision Transformers (ViTs) forces a trade-off between spatial resolution and rollout horizon, particularly for fine-scale PDEs where shocks, reaction fronts, and material interfaces occupy small, evolving regions of the domain. Conventional neural surrogates also lack mechanisms to adapt resolution dynamically. We propose WAMRViT, a ViT that tokenizes inputs as balanced quadtrees using a wavelet-inspired refinement criterion, jointly encodes position and refinement level with 3D rotary positional embeddings, and regrids in cell space during inference for stable long-horizon rollouts. A multi-scale variant retains each leaf at its native source resolution and lets the model learn across resolution levels. Unlike fixed-budget adaptive-tokenization methods, WAMRViT imposes no predetermined token count and supports fully adaptive topology throughout autoregressive rollout. To our knowledge, it is the first machine-learning surrogate to natively tokenize multi-level Adaptive Mesh Refinement (AMR) data. On uniform-grid benchmarks, uniform-patch WAMRViT improves finest-level region-of-interest VRMSE over a finest-patch uniform ViT while using substantially fewer tokens. The multi-scale variant achieves the lowest first-step full-field RMSE and VRMSE on both benchmarks and improves rollout-averaged full-field and refined-region accuracy at long horizons. With parallelized regridding, its end-to-end rollout cost lies between finest-patch and approximately token-matched coarser-patch ViTs. On a complex AMR combustion problem whose finest features cannot be represented natively by the evaluated uniform-grid baselines, WAMRViT operates on adaptive cells built from the simulator hierarchy and substantially reduces finest-level error at matched transformer capacity. Code: https://github.com/tonyzyl/wamrvit

0.39