CS Paper Radar

A four-level executable validation protocol for evaluating LLM-generated topology specifications of physics-based HVAC digital twins

Younghwan Joo, Jeonghoon Han, Sanghyun Oh, Soosik Bang, Sung-il Kim

eess.SY cs.AIcs.LGcs.SE ·submitted 2026-11-19 03:58 UTC · v1

Abstract

Physics-based digital twins of air conditioning, a principal energy consumer in manufacturing and buildings, can be generated automatically from a declarative specification of its equipment, connections, and sizing. Large language models (LLMs) now write such specifications from plain-language descriptions. Evaluations of these specifications stop at parse success, completed runs, or textual similarity, and cannot reject a model that converges in simulation but is physically wrong. In this study, a four-level protocol is proposed that tests whether an LLM-written specification not only executes but also yields a physically valid digital twin. Each specification is executed through static validity, model build, simulation convergence, and behavioral equivalence with a hidden reference model within bands calibrated by $\pm$20% sizing perturbations. The protocol was applied to eight LLMs and fourteen HVAC topologies, five describing an existing low-humidity test chamber. To test for authorship bias, engineers and a second model family also described six topologies. Within three repairs, seven of eight models built in 98.6 to 100% of trials, with simulations converging in at least 80.0%, but reached behavioral equivalence in only 50.0 to 98.6%. About half of the failed trials converged, but not to the reference steady state, mostly in desiccant rotor systems. Pooled pass rates varied by under five percentage points across description authors. These results imply that the physics hardest to model in each field, here the desiccant rotor, must enter the generation framework as validation levels. The protocol can be adapted to other fields that generate executable digital twins from declarative specifications.

Closest papers

Nearest neighbours by meaning across the whole index.

Automated generation of experimentally validated digital twins for desiccant-based low-dew-point air-conditioning systems from declarative topology specifications

eess.SY cs.CE ·2026-08-10 ·Younghwan Joo, Jeonghoon Han, Sang Hyun Oh, Soosik Bang, Sung-il Kim
published similarity 0.67
Abstract

In battery manufacturing, the low-dew-point air conditioning of dry rooms is among the largest energy consumers, and a physics-based digital twin offers insight for operating-point optimization beyond the installed monitoring points. Building one and calibrating it to field data each demand distinct expertise, which limits industrial uptake. We present a framework that generates a dynamic digital twin of an HVAC system from a declarative topology specification, concise enough to draft from a natural-language plant description, compiled against a purpose-built physical component library with wiring, solver, and telemetry synthesized automatically. The models carry equipment-level physics: the desiccant wheel couples heat and mass transfer through an interchangeable sorption-isotherm component, so an undisclosed commercial sorbent is calibrated as an effective isotherm rather than asserted as a material. For experimental validation we built an industrial-grade, ten-component low-dew-point system whose commercial desiccant-wheel unit holds a chamber near -40 °C frost point, and operated it in both dehumidification and bypass regimes. Generation reached a runnable model fifteen times faster than expert manual construction, and a single parameter set, fitted only to three closed-loop humidity nodes, predicts the bypass regime within 0.1 °C, the reactivation-heater power within 5%, and measured input-step responses. Identifiability analysis shows why this is prediction, not fitting: ordinary operating points constrain only one parameter combination, and the deep-dry equilibrium level of the recirculating loop supplies the missing signal. The framework shortens the path from plant description to measurement-validated twin; its criteria-based calibration is a step toward twins calibrated, not only constructed, automatically.

0.52

An Exemplar of a Digital Twin in Mechanical Engineering: Understanding Model Hybridization

cs.SE ·2026-09-15 ·Mahussi Datongnon, Hubert Lejeune, Yoann Jus, Benoit Combemale, Julien Deantoni
top venue similarity 0.56
Abstract

Digital Twins (DTs) are widely adopted across a variety of application domains. In industrial sectors, particularly in mechanical engineering, they accelerate product development, reduce risks, enable early issue prediction, and lower sustainment costs . In practice, DTs increasingly integrate physics-based (deductive) and data-driven (inductive) models into hybrid models combining the complementary strengths of both modeling paradigms. In this paper, we refer to this integration paradigm as hybridization. Despite this trend, the engineering of hybrid DTs that is, remains insufficiently documented. Hybridization is often introduced in an ad hoc manner, and its implementation is only partially made explicit, which limits reproducibility and transferability. This paper reports on the development of an existing fluidic loop digital twin at Centre Technique des Industries M{é}canique (CETIM). The DT is described using the characterization framework of Gil et al., providing a structured view across its lifecycle dimensions. To make hybridization explicit, the case is further analyzed through a complementary characterization structured along two dimensions: motivation and realization. The resulting description provides a traceable account of hybridization decisions and supports the documentation and transfer of hybrid DT engineering practices.

0.46

A System-of-Systems Case Study for the Verification of Composed Digital Twins

cs.SE ·2026-08-28 ·Mennatullah T. Khedr, Mengwei Xu, John S. Fitzgerald, Peter Gorm Larsen
similarity 0.56
Abstract · The 24th Overture Workshop, 18th May 2026

Current approaches to engineering dependable Digital Twins (DTs) of Cyber-Physical Systems lack practical guidance on how qualities such as relevance, verifiability, substitutability and fidelity may be formalised and verified. This need is amplified in Systems-of-Systems (SoS), where reliance is placed on the composition of DTs. The goal of this study is to identify foundational challenges that a framework for DT validation and verification should address in an SoS setting, and to investigate the formalisation of individual DT artefacts as a step towards compositional reasoning. We present a case study based on a DT-enabled greenhouse SoS modelled in VDM-RT (Vienna Development Method, Real-Time), including executable formal models, a property-based account of DT qualities, and an analysis of the obstacles arising when attempting to compose these artefacts at the SoS level. We consider how DT qualities may be operationalised as sets of verifiable properties. Formal modelling and analysis techniques and tools supporting their verification are identified. The study reveals how interpretations of DT qualities must adapt to different architectural roles and how local quality guarantees form assumptions for compositional reasoning. Full formalisation and verification of SoS composition is identified as future work.

0.19

Large Language Models for HVAC Operations in Building Energy Systems: A Critical Review of Methods, Applications, and Deployment Readiness

cs.AI cs.CLeess.SY ·2026-09-04 ·Alexander Neubauer, Tianzhen Hong, Han Li, Mengbo Yu, Amin Darbandi +2 more
survey similarity 0.56
Abstract · 38 pages, 9 figures, 16 tables. Submitted to Energy and Buildings

Building automation systems generate rich sensor data yet remain insight-poor because heterogeneous point naming, missing metadata, and fragmented documentation obstruct their operational use. This systematic review analyses and codes 66 peer-reviewed studies on large language models (LLMs) for HVAC operations published between 2023 and March 2026. Each study is classified across five application families and three LLM method families and assessed for evidence realism, deployment readiness, and the responsibility boundary between the LLM and physical HVAC decisions. The corpus is concentrated in building energy modelling (BEM, 32 of 66 papers), while load forecasting remains too sparse for subfield-level conclusions. Only four studies reach pilot-level evidence, and none reports sustained operational deployment. No study was classified as ready-now for industry adoption; three were near-term and 63 research-only. Nevertheless, several bounded, human-in-the-loop uses merit near-term trials, including point-name normalisation, document-grounded operator support, BEM workflow assistance, and advisory interfaces around physics-based controllers. Conventional machine learning (ML), model predictive control (MPC), reinforcement learning (RL) and ontology-based tools remain more adopted for high-frequency control, short-horizon numerical forecasting, and well-posed ontology mapping, while autonomous agentic operation and unvalidated occupant proxies remain research-stage. Current evidence therefore supports LLMs primarily as semantic and workflow layers rather than autonomous HVAC controllers. Future work should prioritise field-validated benchmarks, orchestration evaluation under operational constraints, and LLM-MPC/RL architectures with bounded latency and verifiable safety properties.

0.64

LLM-enabled Behavior Driven Development Workflow for Formally Verified Hardware Designs

cs.AR cs.SE ·2026-09-14 ·Luca Müller, Qian Liu, Rolf Drechsler
similarity 0.55
Abstract

Recently, the use of Large Language Models (LLMs) for different tasks in the Electronic Design Automation (EDA) life-cycle has been studied extensively, but an integrated view is lacking. Specifications are the foundation of this life-cycle, but they suffer from ambiguity when written in natural language, which especially affects the quality of LLM output. Formal specifications mitigate these ambiguities, but they come with their own challenges. On the other hand, Controlled Natural Language (CNL) specifications can serve as a middle-ground, reducing ambiguity while retaining interpretability. In this work, we propose an integrated view on the use of LLMs for EDA and establish an LLM-enabled behavior driven hardware development workflow. We introduce and define Formal Verification Gherkin Scenarios (FV Gherkin Scenarios), unlocking CNL specifications as the foundation for formally verified hardware designs via Formal Property Verification (FPV). Experimental evaluation shows that our workflow is able to outperform other established LLM-based methods by 2.48x in functional correctness of generated Register Transfer Level (RTL) designs and by 2.54x in formal coverage of generated assertions for FPV.

0.07

SEAL-DT: An End-to-End Security Engineering and Assurance Framework for Cross-Domain Digital Twins

cs.CR ·2026-10-12 ·Iraklis Varlamis, Stelios Zindros, Christos Sardianos, Thomas Lagkas, Panagiotis Sarigiannidis +1 more
similarity 0.54
Abstract

Cyber-physical systems connect physical processes, software, networks and organizations. Securing them requires a clear account of what happened, what was observed, which actions were authorized and what evidence supports each decision. Digital Twin (DT) descriptions often fail to explain how these elements stay consistent, blur the distinction between physical events and uncertain observations, or detach simulation results from their requirements and assumptions. This paper presents SEAL-DT (Security Engineering and Assurance Lifecycle for Digital Twins), a structured approach to building and evaluating secure DTs. It brings entity definitions, process models, deterministic simulations, data contracts, evidence records, automated tests and reproducible results into one connected model. The main contribution is an end-to-end DT security framework, spanning scenario definition, entity/process modelling, simulation, security decision logic, evidence handling, validation and reproducible reporting. Security decisions and simulation results thus trace back to their data, assumptions and requirements. A DT is more than a copy of system state: it states what it can observe, how accurate and current its information is, where it came from and which decisions it may support. We demonstrate SEAL-DT in five scenarios: Car2X protection, passive IoT-device detection, an industrial control surrogate, lawful acquisition in a smart home, and medical-IoT monitoring that preserves continuity of care. Each includes formal models, deterministic and negative experiments, uncertainty analyses, schema-valid fixtures, technical reports and verified manifests, testing multiple configurations under normal and adverse conditions. Results show that the framework supports consistent model construction, traceable execution and comparable internal validation, and that security findings link to observable evidence.

0.20

Can Open-Weight Large Language Models (LLMs) Simulate Human Survey Populations? A Cross-Instrument Calibration Study

cs.AI ·2026-09-26 ·Grandee Lee, Wang Yue
benchmark similarity 0.53
Abstract

Large language models (LLMs) are increasingly used to generate synthetic survey respondents and digital twins of real people, but whether their output preserves real human statistical structure, rather than surface plausibility, remains unresolved, and most existing evidence comes from proprietary models rather than open-weight ones. We evaluate three open-weight LLM families on a cross-instrument calibration task: conditioning personas on real respondents' verbatim answers to one psychometric instrument and measuring them on a second, construct-distance-controlled instrument, checked against a 2,058-person human panel. Across a 139-pair grid, the simulated cross-instrument correlation tracks the real human correlation at r = 0.70 - 0.73 in every model, driven mainly by correct sign rather than precise magnitude and concentrated in pairs of moderate construct distance. A correlation of this magnitude, obtained from untuned open-weight models conditioned only on individual-level survey data, is a substantively encouraging result for LLM-based behavioral simulation and digital-twin applications: specific model families and releases already reproduce a meaningful share of real human cross-instrument structure without any fine-tuning. This capability does not, however, improve monotonically across model releases: on a matched panel, the newest of three tested Llama releases performs worst on two of three headline metrics, so realizing its promise in practice requires release-specific, distance-aware verification rather than a one-time benchmark.

0.17

Mutation Testing of Simulink Cyber-Physical System Models: Challenges and Solutions in Practice

cs.SE ·2026-08-25 ·Murat Kavak, Onur Kilincceker, Serge Demeyer, Kevin Vandenbroucke, Halim Abdurrahman Ceylan
top venue similarity 0.53
Abstract · Accepted for publication at the 41st IEEE/ACM International Conference on Automated Software Engineering (ASE 2026), Munich, Germany, Octobe…

Several experience reports illustrate that mutation testing is capable of supporting a shift-left testing strategy, especially in industries where late bug discovery incurs very high costs and risks. In the context of cyber-physical systems, a shift left implies that engineers need to test the design models used to simulate, prototype, and analyse the feasibility of the system under design. In this paper, we analyse the challenges we encountered and the lessons we learned when incorporating mutation testing in the context of DUCO, a company producing ventilation systems for buildings. The engineers within DUCO have years of experience with model-based engineering centered around Simulink and StateFlow, including unit tests for their models. During a pilot project with our tool prototype MUT4SLX we learned that equivalent mutants, requirement traceability, and mutation testing for Stateflow represent particular challenges not yet reported in the academic literature.

0.33