CS Paper Radar

Geometry-Aware Operator Families for Structured Representation Learning

Zuyuan Zhang, Fei Xu Yu, Tian Lan

cs.LG ·submitted 2026-09-27 01:52 UTC · v1

Abstract

The geometry of latent representations governs which components should interact and how information should propagate, making geometry-aware operator design a fundamental ingredient of structured deep representation learning. However, existing neural architectures typically rely on generic operator templates or geometry-specific constructions, creating a need for a unified framework that can derive admissible operators directly from fixed structural information while remaining adaptive to changing contexts. We introduce \emph{Geometry-Induced Operator Families} (GIOF), a general framework that converts fixed geometry into a structured family of propagation operators and dynamically selects an appropriate member of this family according to the current context. GIOF first transforms geometry-derived interaction channels into reusable generator bases, then combines them through a context-dependent selector and adaptive propagation scale, and finally realizes the selected operator through stable continuous-time propagation and a bottleneck residual layer. We establish theoretical guarantees covering parameter compression, identifiability, stability, locality, compositional structure, and oversmoothing behavior, while controlled experiments validate these mechanisms and experiments on PEMS-BAY and METR-LA achieve the lowest mean MAE across all reported regional-outage settings, improving over the strongest retained baseline by 2.4\%--8.8\% at 30\% missing sensors.

Closest papers

Nearest neighbours by meaning across the whole index.

Where to Compute and How to Interact: Operator-Readable Adaptation with Gauge-Aware Transport

cs.LG ·2026-09-14 ·Zixuan Shen, Quanxu Wan, Bingchuan Wang, Zhi Wang, Biao Luo
similarity 0.65
Abstract

Adaptive meshes enable neural operators for partial differential equations (PDEs) to allocate spatial samples and computation according to local physical structures. Existing approaches, however, mainly address where to compute, with less attention to how information should interact after node relocation. Mesh adaptation changes local sampling scales, neighborhood structures, and geometric contexts, so representations formed at different nodes may not be directly comparable. Direct aggregation can therefore entangle physical variation with discretization-induced representation variation. Because allocation and interaction are jointly optimized through the same output objective, their individual roles are also difficult to distinguish from final errors alone. We introduce operator readability, requiring an adaptive operator to account for and test why computation is allocated to particular locations and how representations interact under the resulting nonuniform discretization. Based on this principle, we propose the Gauge-Aware Adaptive Mesh Neural Operator (GA-AMNO). Physics-informed adaptive allocation answers where to compute, while geometry-conditioned low-rank Gauge transport maps source features into target representation contexts before aggregation, answering how to interact. This makes mesh-to-solver information exchange inspectable and intervenable. We establish sufficient conditions for representation-consistent aggregation and analyze approximate transport errors and continuity under topology-preserving mesh deformations. Experiments on five PDE benchmarks demonstrate improved predictive accuracy, while controlled interventions and geometric-mismatch analyses verify the roles of allocation and interaction and show that Gauge transport improves cross-discretization representation compatibility under strong geometric mismatch.

0.25

Learning structural response operators across varying geometries and loads with global and local geometry encoding

cs.CE ·2026-11-01 ·Xin Liu, Wuwen Yao
benchmark similarity 0.64
Abstract · 27 pages, 20 figures, 7 tables; v2: Removed the template footer; scientific content unchanged

Predicting mechanical responses across complex geometries and varying loading conditions is central to simulation-driven engineering, yet capturing the influence of geometric boundaries remains challenging for data-driven models. Here we propose a Load- and Geometry-adaptive Deep Operator Network (LG-DON) that integrates global geometric representations with local, boundary-aware information to predict structural responses. By distinguishing distances to different boundary categories rather than relying on the nearest boundary alone, the framework captures spatially varying geometric influences that conventional distance-based representations may overlook. Across benchmark problems involving diverse non-convex geometries, LG-DON reduces the mean relative L2 error from 12.17% for a modified MIONet baseline to 7.41%. Controlled ablation studies further demonstrate the advantages of category-resolved boundary distances and show that combining global and local geometric representations outperforms either representation alone in 15 of 18 evaluated configurations. These findings highlight the value of integrating complementary geometric information for accurate operator learning across complex structural domains.

0.34

Dual-Manifold Geometry Guided Representation Learning: Adaptive Coupling between Kernel and Data Spaces

cs.CV ·2026-08-13 ·Wencong Zhang, Yue Zhang, Meiyan Huang, Wei Yang, Qianjin Feng
similarity 0.62
Abstract

Deep representation learning has primarily focused on how features evolve across network layers, while largely overlooking the structured geometry embedded in network parameters. We introduce a dual-manifold perspective in which each convolutional layer contains two coupled geometric spaces: a Kernel Manifold induced by convolutional filters and a Data Manifold characterized by intermediate feature representations. Because these manifolds share the same channel space, parameter geometry can provide complementary structural information to guide feature evolution. Based on this insight, we propose Kernel-Guided Feature Transform (KGFT), a lightweight module that derives a geometric guidance matrix from the kernel Gram matrix and uses it to transform the covariance structure of feature representations. Unlike conventional attention mechanisms that reweight feature responses, KGFT explicitly reshapes feature relationships by transferring geometric information from the kernel manifold to the data manifold. To accommodate network hierarchy, we further introduce Exploit and Explore modes with a depth-aware scheduling strategy and a learnable guidance strength that adaptively controls the contribution of geometric transformation. This design promotes geometric alignment in shallow layers while encouraging feature diversity in deeper layers, without imposing excessive constraints on representation learning. Theoretical analysis establishes the validity of the proposed transformation and characterizes its effect on feature covariance. Extensive experiments across CNN- and Transformer-based architectures, including ResNet, ViT, and LLaMA-7B, demonstrate consistent improvements on image classification and arithmetic reasoning tasks, validating the generality and effectiveness of kernel-guided dual-manifold representation learning. Code will be publicly available.

0.14

Geometric Morphology Learning: Empowering Foundation Models from Structure to Spacetime Intelligence

cs.AI ·2026-11-01 ·Li Sun, Philip S. Yu
similarity 0.62
Abstract · 7 pages, 1 figure, TKDD 20th Anniversary

Geometry is the very form that connects space, time, and matter, serving as the fundamental fabric of the physical world, from continuous fields to discrete structures. Despite its importance, deep learning is inherently blind to geometry. Riemannian geometry stands as a well-established mathematical foundation for studying geometry, yet it remains largely incompatible with deep learning architectures. To bridge this gap, we establish a new learning framework--Geometric Morphology Learning--that learns geometric or topological structures, shapes or conformations for representation learning, reasoning and action modeling. Moving beyond early attempts limited to specific geometric priors, this theory centers on modeling the intrinsic geometry of objects and learning from such geometric or topological knowledge. Specifically, we construct a neural bundle, a unified scaffolding linking local and global structures, and formulate differentiable operators, including connection and curvature, thereby enabling the study of geometry with advanced geometric concepts such as holonomy and cohomology. Consequently, Geometric Morphology Learning opens new horizons in foundation models, reasoning and planning, graph modeling, physical simulation, PDE systems, and dynamics, marking a fundamental shift toward instilling systematic geometry in artificial intelligence.

0.25

CAD-WAM: Anticipating Geometry for Text-to-CAD Generation

cs.CV cs.GR ·2026-10-29 ·Zhengyang Li, Weijie Zheng, Yuxin Cui, Sijie Wang, Wenjie Feng +1 more
code/data similarity 0.61
Abstract

Generating editable computer-aided design (CAD) models from text requires selecting operations that produce the intended changes to the geometry constructed so far. An incorrect operation can compromise subsequent steps, making reliable multi-step generation challenging. Existing methods based on large language models (LLMs) primarily learn to predict commands, without explicit supervision of the geometric states those commands should produce. Inspired by world-action models, we introduce CAD-WAM, which couples geometry-conditioned action generation with next-state prediction. We define complete modeling operations as actions in a CAD domain-specific language (DSL) and encode surface point clouds of intermediate solids into compact latent states using a pretrained variational autoencoder. An LLM backbone generates the next action from the target description and current geometric state, while an auxiliary head on the same backbone predicts the next state to provide geometric supervision during training. We construct step-wise training trajectories from parametric CAD models in the DeepCAD dataset, pairing each trajectory with an expert-level textual description. At inference, each generated action is executed by a CAD kernel, and the resulting solid is re-encoded to inform the next decision. On the DeepCAD test set, CAD-WAM outperforms the task-specific baselines in geometric accuracy and output validity, with larger gains on models requiring more operations, and produces more accurate geometry than general-purpose LLMs prompted with the same DSL. Ablations further show that next-state supervision reduces mean Chamfer distance by 13.2\% relative to an otherwise identical model trained only to predict actions. Code is available at https://github.com/Lzy-1037/CAD-WAM.

0.42

Localized Operator Learning with Adaptive Partition-of-Unity Mixture-of-Expert Networks

cs.LG ·2026-10-03 ·Madison Cooley, Ramansh Sharma, Shandian Zhe, Robert M. Kirby, Varun Shankar
similarity 0.61
Abstract

Operator learning methods such as DeepONets and FNOs often struggle with PDE families featuring sharp interfaces, heterogeneous coefficients, and localized multiscale structures. We introduce a partition-of-unity (POU) mixture-of-experts framework for localized operator learning, in which geometry-aware gating networks produce smooth spatial partitions which blend the contributions of local expert networks. Our main contribution is HiRefPOU, a residual-style hierarchical POU architecture for DeepONets that organizes localized representations through nested parent-child partitions while preserving global continuity. We also show that the same POU principle can be incorporated into Fourier Neural Operators to introduce spatial adaptivity without modifying the underlying spectral layers. On heterogeneous Darcy and reaction-diffusion benchmarks, HiRefPOU achieves substantially lower error than global DeepONet and static POU-MoE baselines, while the broader operator-learning experiments show that the benefits of localization depend on the PDE structure and the chosen neural-operator backbone. The learned partitions are interpretable and align with interfaces and regions of rapid solution variation. These results show that explicit geometric localization can improve both accuracy and interpretability in neural operator learning.

0.26

CascadeJEPA: Enhancing 3D Geometric Representations via Spatially Coupled Cascaded Prediction

cs.CV ·2026-11-17 ·Kailai Zhou, Ziqi Chen, Xiaoning Xu, Yuefeng Wu, Zixuan Fu +6 more
similarity 0.60
Abstract

Joint-Embedding Predictive Architectures (JEPAs) have emerged as a promising paradigm for predictive visual representation learning. However, our analysis of I-JEPA reveals a growing discrepancy between semantic and geometric learning: improved semantic recognition during pretraining is accompanied by degraded 3D geometric representations. We hypothesize that this partly stems from predicting highly abstract latent representations with incomplete spatial context, favoring semantic regularities over fine-grained geometric information. To address this issue, we introduce CascadeJEPA, a spatially coupled cascaded prediction framework that promotes richer patch-level interactions through full-grid context encoding and cross-attention between prediction stages, while largely preserving the original I-JEPA architecture. Extensive experiments demonstrate substantial improvements over I-JEPA across multiple 3D geometric tasks while maintaining strong semantic recognition performance. Furthermore, its advantages extend to future feature forecasting and world model planning, highlighting the potential of enhanced geometric representations to support predictive world modeling.

0.15

FG-ConvNO: A Geometry-Aware Neural Operator for Propeller CFD Prediction

cs.LG ·2026-10-24 ·Yichen Di, Valentin Duruisseaux, Di Zhou, Xinyi Li, Daniel Leibovici +2 more
publishedhot direction similarity 0.60
Abstract · 18 pages, 5 figures. Presented at the AAAI 2026 AI2ASE Workshop (non-archival)

Computational Fluid Dynamics (CFD) is essential in contemporary engineering design, yet it remains expensive to compute and can take thousands of CPU/GPU hours or more for a single high-fidelity simulation. Recent developments in machine learning for scientific modeling have led to powerful surrogate frameworks that can approximate flow fields and physical quantities by using a fraction of the computational cost. However, most of these methods require largescale homogeneous datasets or simple geometries, exhibiting severe generalization gaps when applied to complex and data-scarce configurations. To address this, we propose the Factorized Geometry Convolutional Neural Operator (FG-ConvNO) for robust geometry-aware propeller pressure prediction under limited training data. We introduce geometryaware encoding, efficient interpolation-based decoding, and auxiliary global quantity supervision. In addition, we incorporate DISCO blocks that use discrete continuous convolutions to make the model discretization agnostic. In our experiments, FG-ConvNO achieves the highest accuracy for both surface pressure and global thrust prediction on a procedurally generated propeller CFD dataset. Geometry-aware encoding and architectural simplification improve stability, while random vertex subsampling enhances robustness. Overall, adding geometric priors to grid-based convolutional operators reduces overfitting on complex propeller geometries and improves generalization in low-data regimes.

0.61