CS Paper Radar

LLMs as Master Forgers: Generating Synthetic Time Series Data for Manufacturing

Mantek Singh, Jeshwanth Challagundla, Prateek Karnal, Gagan Ganapathy, Vineet Shah, Ridam Arora

cs.LG cs.AI ·submitted 2026-09-14 18:01 UTC · v1
top venuehighly cited

Abstract

This paper presents a novel framework leveraging Large Language Models (LLMs) to generate synthetic time series data for manufacturing processes. Motivated by the scarcity of labeled time-series data in real-world manufacturing settings, which hinders the development of robust machine learning models, we explore the potential of LLMs to learn complex temporal dependencies and generate realistic synthetic data. Our approach involves fine-tuning pre-trained LLMs on manufacturing process instructions and employing a Retrieval Augmented Generation (RAG) technique to enhance data diversity and realism. We evaluate our method against traditional time series modeling techniques like ARIMA and LSTMs, using quantitative metrics, PCA analysis, and downstream task performance (anomaly detection). Results demonstrate that our LLM-driven framework outperforms these baselines, generating high-quality synthetic time series data that effectively captures temporal dependencies and statistical properties of real manufacturing data, leading to improvements in downstream task performance.

Comment
7 pages, 4 figures. Published in the 2024 International Conference on Image Processing, Computer Vision and Machine Learning (ICICML)
Journal ref
2024 International Conference on Image Processing, Computer Vision and Machine Learning (ICICML), pp. 2053-2059, IEEE, 2024
DOI
10.1109/ICICML63543.2024.10958017

Closest papers

Nearest neighbours by meaning across the whole index.

Streaming Virtual Metrology via Lightweight Adaptation under Temporal Distribution Shifts

cs.LG ·2026-10-31 ·Doyun Choi, Cheonwoo Lee, Younghyun Han, Kiwook Han, Jinkook Park +3 more
published similarity 0.67
Abstract · Accepted at IEEE BigData 2026. 10pages

Virtual metrology (VM) aims to estimate costly or delayed metrology variables from heterogeneous sensor signals and in-line measurements, and it acts as a critical component of semiconductor manufacturing. While recent data-driven models have improved VM accuracy, most are trained in batch settings and assume stationary data distributions. In practice, however, production environments evolve continuously due to equipment degradation, process adjustments, and maintenance events, making conventional batch-trained VM models difficult to maintain under temporal distribution shifts. In this work, we propose GRAPE (Global Regression with Adaptive Prediction Enhancement), a practical online learning framework for VM in evolving wafer-processing environments. GRAPE integrates a stable global predictor based on interpretable representations with two lightweight streaming adapters that correct feature-level and environment-level shifts through online updates. Experiments on two real-world semiconductor manufacturing datasets, including chronologically held-out blind-test streams, show that GRAPE improves robustness and generalization under real temporal drift compared with both conventional batch-learning methods and existing online learning baselines.

0.37

MouldIt: Data Efficiency of Multi-Task Temporal Representation Learning in Injection Moulding

cs.LG cs.ROeess.SP ·2026-11-10 ·Abdullah Ayad
similarity 0.67
Abstract · 14 pages, 10 figures, 4 tables

Smart industrial solutions that can address multiple tasks require models that understand the underlying processes. We examine representation learning for modeling the injection moulding process as an example of such solutions. We consider its data efficiency and multi-modal benefit compared to classical machine learning alternatives. To this end, we synthesize a process-informed dataset based on a real reference cycle of an Erlanger Träger part. We then introduce MouldIt, a temporal convolution and attention based variational autoencoder that reconstructs in-mould signals, while classifying parts as acceptable or as one of our synthetic failure modes. We evaluate MouldIt on two tasks, known-class classification and unknown-class detection, across nine dataset sizes in single-modal and multi-modal settings. We compare it with an accept-all baseline and the classical methods k-nearest neighbors, logistic regression, random forests, and principal component analysis, which we adapt to support both tasks. MouldIt outperforms classical methods across both tasks from 4000 cycles in the single-modal setting and from 2000 cycles in the multi-modal setting. It achieves part sorting losses of 6.0 $\pm$ 0.7% and 1.0 $\pm$ 0.3% in the single-modal and multi-modal settings respectively, ranking first across all baselines and a combination of principal component analysis and random forests.

0.22

Process-Aware Event Log Augmentation for Process Prediction under Labeled Data Scarcity

cs.LG ·2026-10-17 ·Sjoerd van Straten, Alessandro Padella, Marwan Hassani
publishedcode/data similarity 0.66
Abstract · This paper has been accepted for the International Journal of Cooperative Information Systems

Predictive Process Monitoring (PPM) forecasts future events of ongoing business processes based on event logs. However, deep learning PPM approaches are often limited by labeled data scarcity and the low variability of real-world logs. To address this, we introduce SiamSA-PPM, a self-supervised framework combining Siamese learning with statistically grounded data augmentation. It employs novel transformation methods that leverage control-flow semantics and frequent behavioral patterns to generate realistic, semantically valid trace variants, learning generalizable prefix representations without labeled supervision. Extensive experiments on real-world event logs with a wide range of variability, incorporating exhaustive hyperparameter tuning, demonstrate the value of this process-aware augmentation. Compared to state-of-the-art baselines, SiamSA-PPM achieves competitive or superior predictive performance in both next activity and final outcome prediction tasks. An open source implementation of our method is publicly available under https://github.com/SvStraten/SiamSA-PPM-Extension..

0.37

D-TAIA: Domain-Aware LLM Adaptation for Multi-Task Predictive Process Monitoring

cs.LG ·2026-08-28 ·Sjoerd van Straten, Christine Jacob, Marwan Hassani
similarity 0.64
Abstract · This paper has been accepted for the AI4PM Workshop at ECML PKDD 2026 in Naples, Italy

Predictive Process Monitoring (PPM) enables organizations to forecast future process behavior, such as the next activity and remaining time of ongoing cases. In practice, three conditions cause existing methods to degrade, namely data scarcity, high process entropy and distributional shift. While Foundation Models (FMs), especially Large Language Models (LLMs), offer a new paradigm through broad sequential reasoning, adapting them to multi-task PPM under these conditions remains an open challenge. Existing FM-based approaches either lack mechanisms for handling distributional shift or rely on direct regression heads that can be structurally misaligned with continuous time prediction tasks. This paper introduces D-TAIA (Domain-aware Training and Attention-based Inference Architecture), a framework for a joint next activity and remaining time prediction task via parameter-efficient fine-tuning of an FM backbone. Our approach combines domain-aware triplet loss (DATL) pre-training with FAISS-based nearest neighbor retrieval for remaining time prediction, and adopts the TAIA inference strategy to preserve pre-trained sequential reasoning during fine-tuning. Evaluated across four real-world event logs, D-TAIA consistently shows SOTA or competitive performance compared to a fine-tuned LLM and a recurrent neural network baseline. Ablation studies confirm that techniques from NLP and computer vision can be transferred effectively to PPM with only a 10M-parameter backbone, though component contributions vary by dataset entropy.

0.52

CAST: Context- and Anomaly Structure-Conditioned Time Series Anomaly Generation

cs.LG ·2026-08-19 ·Haochen Zhang, Jie Peng, Songyuan Sui, Yu-Chao Huang, Xiangqi Zhu +1 more
similarity 0.64
Abstract

Anomalous time series play a critical role in safety-critical domains, yet they are inherently scarce, heterogeneous, and costly to obtain. Existing time series generation methods predominantly focus on synthesizing normal data, providing limited value when anomalous samples are needed. We identify two fundamental challenges in anomaly generation: (i) the scarcity of anomaly data, and (ii) the heterogeneous morphological characteristics of anomalies. To address these challenges, we propose CAST, a Context- and Anomaly Structure-conditioned Time series anomaly generation framework with principled two-stage pretraining and finetuning strategy. In pretraining stage, we leverage abundant normal time series data to learn underlying system dynamics and substantially mitigate the limited availability of anomaly data. During finetuning, CAST explicitly conditions the generator on learned anomaly structure representations, enabling it to capture heterogeneous anomaly morphologies under similar contextual conditions. Extensive experiments on multiple real-world univariate and multivariate datasets demonstrate that CAST consistently outperforms state-of-the-art anomaly generation methods in terms of both generation fidelity and downstream task utility, highlighting the effectiveness of the proposed approach.

0.04

Lotus: Learning to Understand Time-series by Learning to Forecast

cs.LG cs.AIcs.CL ·2026-10-01 ·Van Dai Do, Huu Hiep Nguyen, Minh Hoang Nguyen, Manh Nguyen, Hung Le
code/data similarity 0.64
Abstract

Time-series language models typically align numerical signals with language using labels, captions, or question-answer pairs, without explicitly requiring their representations to preserve how the underlying signals evolve. We introduce forecast grounding, a pretraining principle that requires language-aligned time-series representations to remain predictive of the future evolution before they are adapted to downstream tasks. We instantiate this principle in Lotus, which divides each time-series channel into non-overlapping patches and maps them into the LLM embedding space. It then predicts the next patch from the LLM hidden states of the time-series tokens, encouraging the learned representation to retain temporal information. To capture both temporal changes and periodic patterns, we also propose a temporal-spectral patch representation that combines each raw time-series patch with its spectral representation before mapping it into the LLM embedding space. Across three language model backbones and six downstream evaluation datasets, Lotus achieves the best average performance with minimal trainable parameters, reaching 59.91% accuracy and outperforming the strongest matched baseline by 3.36%. Our representation analyses further show that Lotus uses the numerical input and preserves temporal structure in its learned patch representations. These results establish forecast grounding as a promising pretraining principle, enabling compact time-series language models to outperform models 25-50 times their size on TSRBench. Our code is publicly available at https://github.com/DA2I2-SLM/Lotus

0.30

ChronoSSM: Training for Temporally Aware Representations in Autoregressive State Space Models

cs.LG cs.NI ·2026-08-10 ·Adrien Schoen, Nachiketa Ratnakar Patil, Arjun Bhagoji, Francesco Bronzino
similarity 0.64
Abstract

Modern sequence models, from Transformers to State Space Models, have enabled powerful generative modeling across diverse domains, yet they are typically trained to predict what happens while treating when it happens as a secondary concern. In data-mining settings where events are associated with explicit timing information, this separation can limit temporal reasoning, anomaly detection, and faithful reconstruction of event chronology. A common strategy is to treat timing as an auxiliary signal, training a separate timing model using representations learned solely for event prediction. However, this two-stage approach implicitly assumes that representations optimized for event prediction already contain sufficient temporal structure. We introduce ChronoSSM, an autoregressive State Space Model (SSM) that jointly models events and timestamps with a shared backbone trained using combined token and temporal generation objectives. We compare the joint regime, where temporal supervision updates the backbone, with the two-stage regime, where timing is learned only using the frozen event representations. Across four domains spanning dense and partial timestamp supervision, joint training consistently makes inter-arrival information more recoverable from frozen representations without any systematic degradation in content-generation quality overall. Our results show that temporal supervision can produce more temporally informative representations without materially degrading autoregressive event modeling.

0.04

A Data-Driven Framework for Unsupervised Monitoring of Transmission Systems Using End-of-Line Testing Data: A Case Study at Ford Motor Company

cs.LG stat.APstat.CO ·2026-10-04 ·Mohammad N. Bisheh, Mehrdad Moradi, Parinaz Farajiparvar, Colin Brady, Rajesh Gupta +4 more
similarity 0.64
Abstract

Sensing technologies have advanced rapidly across industries ranging from energy to automotive manufacturing. These systems generate high-dimensional (HD) data characterized by complex nonlinear patterns and strong temporal dependencies. Traditional statistical monitoring methods are often limited in their ability to capture such nonlinear structure. Likewise, many analytical approaches used in End-of-Line testing rely on predefined thresholds and heuristic rules, which restrict their ability to detect informative anomaly signatures in HD temporal data. In contrast, while modern deep learning and generative AI models offer strong predictive capabilities, they are often unsuitable in applications where data are costly to collect and where the monitoring system must remain interpretable, low-latency, computationally efficient, and usable by non-technical practitioners. To overcome these limitations, we propose an advanced multivariate monitoring framework for HD data. The framework operates in two stages. In the first stage, the data are preprocessed to remove incomplete and non-informative samples and to temporally align time series data. In the second stage, nonlinear dimensionality reduction is performed, followed by anomaly detection through a control chart based phase I monitoring procedure. The framework can be used in both unsupervised and supervised settings, depending on the availability of ground truth labels during training. Moreover, its flexible and modular structure allows practitioners to adapt its components to different domains and operational requirements. We evaluate the proposed framework on real production data from an automotive manufacturing environment at Ford Motor Company. The proposed method achieves higher accuracy, recall, and F1 score than the company's existing model, improving these metrics from 0.50, 0.30, and 0.429 to 0.625, 1.00, and 0.769, respectively.

0.23