CS Paper Radar

Bilevel Optimization of Topology and Hyperparameters (BOTH)

Suryanarayanan Manoj Sanu, Miguel Anibal Bessa, Alejandro Marcos Aragón

cs.LG ·submitted 2026-09-18 13:29 UTC · v1

Abstract

Topology optimization (TO) represents a significant step towards automating the design process: given a working simulation, TO can produce a viable prototype at the press of a button by differentiating the simulation and iteratively improving the design. In practice, however, TO is riddled with ``magic numbers''---hyperparameters whose tuning significantly affects the outcome. Finding the right values typically requires not only deep problem-specific knowledge but also extensive trial-and-error. While practitioners can use surrogate-assisted hyperparameter optimization as an alternative, this approach requires strictly limiting the number of hyperparameters through careful problem formulation. Here, we propose differentiating TO itself using automatic differentiation. This yields ``hypergradients'' that allow us to tune these hyperparameters in tandem with the primary optimization. We show that evaluating just one or two steps of TO is sufficiently informative and that the method scales favorably to thousands of hyperparameters at an expense comparable to only a few standard TO runs. We demonstrate this approach on stress-constrained and compliance problems, with the latter utilizing a neural parameterization of the density field.

Comment
Currently under submission to SMO journal

Closest papers

Nearest neighbours by meaning across the whole index.

Scalable Stress-Constrained Topology Optimization: A Simultaneous Approach Based on Machine Learning

cs.LG ·2026-10-20 ·Xiangyu Sun, Shirin Hosseinmardi, Ramin Bostanabad
similarity 0.67
Abstract · 41 pages, 19 figures, journal paper

Large-scale topology optimization (TO) remains computationally demanding, especially for stress-constrained problems. To address this challenge, we propose the simultaneous neural topology optimization (SNTO) framework where state, adjoint, and design variables are represented with custom neural networks (NNs) whose parameters are jointly optimized within a single loop using adjoint analysis and automatic differentiation. The loss function combines the design objective, an aggregation of the local stress constraints, and the potential energy functionals. Our NNs have a unique architecture that accelerates convergence while capturing the sharp gradients and boundaries which are critical to stress-constrained TO. To scale to millions of degrees of freedom, we develop a hardware-aware training scheme in which the design domain is partitioned into subdomains distributed across multiple graphics processing units (GPUs). To control peak GPU memory, we further slice each subdomain into smaller regions whose loss gradients are evaluated sequentially and accumulated. Unlike domain decomposition, our partitioning naturally ensures field continuity across subdomains without additional loss terms. We first evaluate SNTO on two-dimensional (2D) stress-constrained benchmarks and compare the results to a conventional finite element-based approach, showing that SNTO produces designs with sharper boundaries and smaller gray areas. We then solve three-dimensional (3D) problems with up to eight million collocation points on eight GPUs. The training time decreases nearly in proportion to the number of GPUs and the peak GPU memory is reduced by increasing the number of sub-slices. Overall, SNTO offers an effective pathway to large-scale stress-constrained TO on limited computational resources.

0.24

KATOsuper: Surrogate-accelerated neural topology optimization with sensitivity-consistent Fourier neural operators

cs.CE cs.AIcs.LGmath.NA ·2026-09-23 ·Shengyu Yan, Jasmin Jelovica
benchmark similarity 0.63
Abstract · 32 pages, 24 figures, 7 tables

Topology optimization (TO) remains computationally intensive due to repeated finite element analysis (FEA) evaluations required at each iteration. While neural network-based surrogates offer potential acceleration, existing approaches often suffer from gradient inconsistency between predicted objectives and sensitivities, leading to optimization instability. This work presents KATOsuper, an objective-agnostic framework that couples neural-reparameterized topology optimization with a Sensitivity-Consistent Fourier Neural Operator (SC-FNO). The framework employs the forward_split architecture, which derives deployed sensitivities via automatic differentiation through the predicted objective field and thereby preserves consistency between the predicted objective and the gradient used for optimization. The case studies include three 2D benchmark problems and three 3D structures considering compliance or stress minimization. A physics-informed multi-channel input encoding with Fourier position embedding enables resolution-invariant learning, supporting zero-shot extrapolation beyond the training resolution, with useful performance at moderate scaling factors and topology-preserving exploration at up to 64x without retraining. The framework extends to 3D through KATO3D, featuring novel KANConv3D blocks with learnable B-spline activations. KATOsuper demonstrates 15--110x deployment-time speedup over MATLAB baselines while maintaining competitive optimality, with the clearest gains observed in complex 3D and stress-optimization cases. The insight that sensitivity direction matters more than magnitude enables robust optimization even with approximate physics evaluation, extensible to other differentiable physics-driven design objectives.

0.58

KPI-Conditioned Generative Design of Automotive Hood Inner Panels: A Two-Stage Retrieval-Generation Pipeline with Surrogate-Based Performance Estimation

cs.LG ·2026-08-23 ·Sudeep Chavare
similarity 0.63
Abstract · 20 pages, 8 figures, 9 tables

An inner hood panel must meet a deflection target, stay below a stress limit, and hit a mass target. Machine-learned surrogates have made the forward direction, geometry to performance, fast and routine. The inverse direction, producing geometry from a stated requirement, remains largely unaddressed for industrial parts whose design space is organized into discrete topology families rather than a continuous parameterization. This work presents a two-stage pipeline for that inverse problem. A reachability stage determines which topology families can satisfy a given requirement vector. A conditional variational autoencoder then generates point-cloud geometry within a selected family, and a neural-operator surrogate estimates the performance of each candidate. The pipeline is built entirely from public data and freely available compute, and is deployed as an interactive tool. The pipeline works, with qualifications that are reported as primary findings rather than caveats. The surrogate is accurate in aggregate, but its error is comparable to the performance differences it is asked to discriminate, which bounds what can be claimed for any individual generated design. That ratio of surrogate error to within-class signal is argued to be the quantity that determines whether a pipeline of this kind can work at all.

0.40

BOCoDe: Engineering-Centered Benchmarking for Bayesian Optimization

cs.CE ·2026-08-15 ·Rosen Ting-Ying Yu, Christophe Hatterer, Advaith Narayanan, Cyril Picard, Faez Ahmed
benchmarkcode/data similarity 0.62
Abstract

Bayesian optimization (BO) is a sample-efficient, surrogate-based approach to black-box optimization (BBO), but its evaluation remains dominated by synthetic functions and hyperparameter optimization (HPO) tasks that are typically low-dimensional and single-objective. Engineering design poses a substantially different regime: problems are physics-based, often high-dimensional, constrained by requirements such as cost and manufacturability, and may involve multiple objectives or mixed variables. To close this benchmarking gap, we introduce BOCoDe, an open-source, PyTorch-native benchmark comprising 307 BBO problems, including 159 engineering design tasks and widely used synthetic and HPO benchmarks. Each problem includes cited provenance and machine-readable metadata that supports programmatic discovery, including by LLM-based agents, and all tasks are exposed through a unified API compatible with open-source BO libraries. We evaluate 31 BO and evolutionary algorithms across five problem classes spanning single- and multi-objective optimization, constrained and unconstrained settings, and mixed-variable search spaces. Analyses of problem structure show that engineering tasks uniquely span constrained and multi-objective settings that synthetic and HPO suites rarely cover, while embeddings from a tabular foundation model separate them most clearly from HPO tasks. Algorithm rankings also vary substantially across domains; in several problem classes, rankings obtained on standard benchmarks do not transfer to engineering tasks. BOCoDe establishes a reproducible and extensible foundation for developing and evaluating BO methods that better reflect the demands of engineering design. Code & data can be found at https://github.com/rosenyu304/BOCoDe

0.58

Benchmarking Tabular Foundation Models as Surrogates in Expensive Evolutionary Optimization

cs.NE cs.LG ·2026-09-16 ·Lu Han, Jin Wang, Yuchen Li, Haoran Gu, Shulei Liu +3 more
similarity 0.60
Abstract

Surrogate-assisted evolutionary algorithms (SAEAs) are effective methods for solving expensive optimization problems (EOPs), where surrogate models replace most expensive evaluations and critically influence the final optimization results. In recent years, tabular foundation models have advanced rapidly, and the Tabular Prior-data Fitted Network (TabPFN) has been adopted as a surrogate model for EOPs due to its strong predictive capability, demonstrating promising performance. Motivated by its potential as a surrogate model in SAEAs, this work conducts a comprehensive study that combines extensive experiments with in-depth theoretical analysis to investigate the effectiveness of TabPFN. Specifically, we perform experiments across both offline and online SAEA settings, covering diverse problem scenarios such as single-objective, multi-objective, constrained, combinatorial, mixed-variable, and engineering optimization problems. In addition, we further analyze the advantages and limitations of TabPFN within SAEAs and provide practical guidelines for its application in different optimization settings. Results show that the effectiveness of TabPFN is highly problem dependent, and it cannot replace conventional surrogates universally. Overall, TabPFN should be adopted selectively according to data availability, landscape complexity, search space characteristics, and its role within the algorithm. Customized model management strategies and role-specific algorithm design are necessary to fully exploit its advantages and avoid its pitfalls.

0.25

GNBG-C: A Unified and Configurable Benchmark Generator for Constrained Continuous Optimization

cs.NE math.OC ·2026-10-27 ·Ali Ghasemzadeh, Erfan Babakhani, Armin Khosravi, Pariya Ghasemzadeh, Yong-Hong Kuo +2 more
benchmark similarity 0.60
Abstract

Many real-world optimization tasks are governed not only by what should be optimized, but also by what is allowed: a solution must first satisfy physical, operational, safety, budgetary, or structural requirements before its objective value becomes meaningful. Yet benchmark design for constrained continuous optimization often treats these restrictions as fixed properties of predefined test instances. Existing generators and benchmark suites provide essential foundations for reproducible comparison, but they either focus primarily on configurable objective landscapes or expose constraint properties within limited, predefined regimes, making it difficult to ask targeted questions about constraint-induced difficulty: whether an optimizer struggles because feasible regions are small, boundaries are restrictive, variables are tightly coupled, or constraint information is poorly scaled. We introduce GNBG-C, a Generalized Numerical Benchmark Generator for Constrained Continuous Optimization. The central idea is to treat the feasible region itself as a configurable object rather than as an inseparable attachment to a fixed test function. By generating controlled constraint regimes around the known GNBG global minimizer, the generator enables systematic studies in which feasibility pressure can be varied while preserving that point as a global minimizer of the constrained problem. This allows researchers to examine how optimizers discover feasibility, handle constraint pressure, and improve objective quality under diverse and controllable feasibility conditions. GNBG-C therefore provides a flexible and auditable framework for probing the strengths and weaknesses of constrained continuous optimizers under targeted forms of feasibility-side difficulty.

0.20

How the Classical Baseline Shapes Machine-Learning Speed Claims in Topology Optimization

cs.CE cs.LG ·2026-10-12 ·Shaoliang Yang, Jun Wang
similarity 0.60
Abstract · 32 pages, 8 figures, 10 tables. Research article: tests of released code for eight topology-optimization methods and an exploratory audit of…

Topology optimization places material in a design domain to optimize a structure's performance, such as its stiffness or heat conduction. Machine-learning methods often report large speed-ups over classical, non-learned optimization, but a slow classical code or a lower-quality learned design can exaggerate them. We re-ran released code for eight methods on one workstation and studied two in depth: Optimize Any Topology (OAT), a diffusion model that generates designs, and a graph-based neural optimizer. Replacing the classical code with an efficient implementation of the same algorithm cut these two methods' speed-ups by up to 20-fold. We then timed the classical code until its design was at least as stiff as the learned one; for OAT, we picked this point after the runs, which gives the classical code its best case. Published timings imply a 134x speed-up for OAT on a 256 x 256 grid; our matched single-design comparisons gave 1.5x to 3.5x. At matched stiffness, the graph-based optimizer was slower than the classical code in all six two-dimensional problems. Of the six other methods, two kept their advantage, one shrank, one reversed, and two were not comparable. In an exploratory audit, language models judged that 8 of 28 speed comparisons in randomly sampled, openly accessible articles lacked evidence of equal design quality; our own review counted 11, so the count depends on who judges. We provide a comparison procedure and a minimum report: classical code, hardware, design quality, what each timing includes, and data-generation and training costs.

0.27

ATLAS: Adaptive TDA-guided Landscape-Aware Transistor Sizing

cs.AR ·2026-10-07 ·Youngmin Oh, Jihwan Won, Yuntae Park, Bosun Hwang, Suwan Kim
code/data similarity 0.59
Abstract

Analog transistor sizing, finding design parameters that simultaneously satisfy multiple performance specifications, is a labor-intensive bottleneck in circuit design. To support analog circuit experts, various automation methods have been proposed, including Bayesian optimization (BO), reinforcement learning (RL), and others. Yet existing methods are oblivious to the topological structure of a feasible design space, which can fragment into disconnected regions due to operating-regime transitions, conflicting specification trade-offs, and nonconvex device physics. This topological blindness causes the optimizer to converge within a single feasible region while missing others that may contain superior designs. To address this limitation, we propose ATLAS. a BO framework utilizing Topological Data Analysis (TDA). At each iteration, a Mapper graph is constructed over a surrogate-predicted feasible region to estimate connected regions, enabling topology-aware exploration from the very first iteration without any observed feasible points. A topological sensitivity score classifies candidates as bridge, frontier, or interior points, injecting a targeted exploration bonus into the acquisition function. Experiments on four analog circuit benchmarks in the GF180 and SKY130 processes demonstrate that \coin finds feasible designs with significantly fewer simulations than baselines, including RL and BO methods. To the best of our knowledge, this is the first work to apply topological data analysis to analog circuit design automation. The official implementation is publicly available on https://github.com/youngmin0oh/atlas.

0.44