CS Paper Radar

Reinforcement Learning-Based Production Scheduling in an Industry-Based Coating Scenario Using the Digital Model Playground

Arne Kröger, Ralf Buschermöhle, Wilhelm Hasselbring, Henrik Wilbers

cs.AI ·submitted 2026-08-14 09:28 UTC · v1
publishedbenchmarkcode/data

Abstract

Production scheduling in complex manufacturing environments is challenging when sequence-dependent setup times, stochastic disturbances, and due-date constraints must be addressed simultaneously. While reinforcement learning (RL) methods have shown promising results in research, most studies rely on simplified benchmark processes, limiting their industrial relevance. This paper demonstrates the applicability of RL-based scheduling in an industry-inspired coating process that reflects practical complexities such as sequence-dependent setup times, machine breakdowns, and variable utilization. The open-source Digital Model Playground (DMPG), a discrete event simulation framework, is used to model the scenario and to train RL agents. Two standard algorithms, Deep Q-Networks and Proximal Policy Optimization, are benchmarked against conventional dispatching rules to illustrate feasibility and to provide a transparent testbed for further research. Results indicate that RL-based scheduling achieves balanced improvements across key performance indicators, with PPO delivering the most robust performance. The main contribution of this work is to bridge the gap between academic research and industrial practice by validating RL-based scheduling in a realistic, shareable scenario and by providing a reusable open-source framework for future studies.

Comment
Copyright 2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works
DOI
10.1109/ICECET65726.2026.11632318

Closest papers

Nearest neighbours by meaning across the whole index.

Dual Representation Enhanced Diffusion Policy for Flexible Job Shop Scheduling

cs.LG cs.AI ·2026-11-11 ·Yunyi Zhao, Wei Zhang, Cheng Xiang, Vadakkepat Prahlad, Zuming Liu
similarity 0.72
Abstract

The Flexible Job-Shop Scheduling Problem (FJSP) is a fundamental and challenging optimization problem in smart manufacturing. Although recent deep reinforcement learning (DRL) approaches can learn effective scheduling policies through interaction with the environment, their performance often degrades as problem scale and processing-time uncertainty increase. In this paper, we propose DredRL, a dual representation enhanced diffusion-policy reinforcement learning scheduler. DredRL first constructs enhanced dual representations of operations and machines through self- and cross-attention mechanisms to capture both intra-entity and inter-entity dependencies. A diffusion model is further employed as the policy network, generating scheduling decisions through iterative denoising conditioned on the encoded state to enhance policy expressiveness and robustness. Extensive experiments on synthetic, public, large-scale, and stochastic FJSP benchmarks demonstrate that DredRL outperforms state-of-the-art learning-based schedulers across most configurations. Compared with SOTA learning-based methods, DredRL reduces makespan by up to 5.44% on synthetic instances. It also achieves the best learning-based performance on three of four public benchmarks and outperforms the stochasticity-aware algorithms on stochastic benchmarks. These results demonstrate that combining enhanced dual representations with diffusion-based policies provides an effective and robust approach for large scale scalable FJSP scheduling in complicated environment.

0.08

PORL: Pretrained Offline Reinforcement Learning for the Job Shop Scheduling Problem

cs.LG cs.AI ·2026-09-25 ·Mateo Toro Diz, Jonathan Hoss, Noah Klarmann
publishedhot direction similarity 0.72
Abstract · This paper has been accepted for presentation at the IEEE 10th International Conference on Computational Systems and Information Technology …

The Job Shop Scheduling Problem (JSSP) is a fundamental combinatorial optimization problem in industrial optimization. This work introduces Pretrained Offline Reinforcement Learning (PORL), a hybrid approach that combines simulation-based online pretraining with offline fine-tuning on production-specific data. Reinforcement learning through online interaction enables exploration of general scheduling strategies, but typically relies on simulation environments and may suffer from a simulation-to-reality gap. In contrast, offline RL avoids direct interaction with the environment by learning from historical data, but its performance is strongly influenced by dataset quality and coverage. PORL combines the strengths of both paradigms by first learning a general scheduling policy through online interaction and subsequently adapting it offline to a target distribution. A KL-divergence-based policy constraint is introduced to limit deviations from the pretrained policy during fine-tuning. The approach is evaluated on JSSP instances with distribution shift and datasets generated from heuristic, noisy-expert, and random behavioral policies. The results show that PORL consistently achieves lower optimality gaps than standalone offline RL and the considered general scheduling baselines. Furthermore, its advantage over standalone offline RL increases as dataset quality decreases, indicating reduced sensitivity to the quality and coverage of the available offline data. The results suggest that offline adaptation of pretrained policies is a promising approach for industrial scheduling environments where direct online exploration is impractical.

0.34

Distributed Optimization of Modular Production Systems using Model-based Reinforcement Learning with Inverse Models

cs.AI cs.LGeess.SY ·2026-09-10 ·Andreas Schwung, Steve Yuwono, Sofiene Lassoued, Dorothea Schwung
similarity 0.68
Abstract

This paper presents a novel approach for data-driven self-learning control of highly flexible, modular manufacturing systems. Specifically, we employ a novel framework for model-based reinforcement learning which introduces approximate inverse process models within the training of reinforcement policies. This approach disentangles the learning of actuation dynamics and the dynamics in state space, resulting in RL-based training solely within the task space. We propose a lightweight feedforward architecture for approximate inverse models and integrate them within the policy network of standard RL algorithms. We apply the approach to a laboratory modular production testbed with heterogeneous production modules. The results underline the efficiency improvements for modular manufacturing units in terms of both performance and training speed, particularly for off-policy algorithms.

0.17

Augmenting PID Control with Deep Reinforcement Learning: A Hybrid Approach to the Industrial Benchmark

cs.LG ·2026-09-18 ·Zhengyang, Gu, Joseph E. Hernandez, John Burtenshaw, Sean Scott +2 more
top venuebenchmark similarity 0.66
Abstract · Published in: 2026 7th International Conference on Artificial Intelligence, Robotics, and Control (AIRC)

As industrial processes grow in complexity, traditional Proportional-Integral-Derivative (PID) controllers are often insufficient for handling their non-linear, multi-input dynamics. We propose using advanced Deep Reinforcement Learning (DRL) to prove its advantages in these complex environments. To do this, we rely on the Industrial Benchmark (IB). The IB is a realistic simulation that tests DRL algorithms against the key challenges of industrial applications: high-dimensional state spaces, delayed effects, and conflicting multi-criterial objectives. This testbed highlights DRL's core trade-off: while its final policies can often be unstable, its unique strength is the ability to autonomously discover optimal, non-obvious policies in multi-dimensional spaces where simple controllers fail. In this paper, we propose a novel hybrid PID-RL controller that leverages DRL's discovery capability while ensuring Reliability. After developing a multi-objective reward function to make DRL viable, we use a twin-delayed deep deterministic (TD3) agent as a discovery tool to find the optimal, non-obvious settings for the IB's 'Gain' and 'Shift' parameters. By feeding these discovered parameters to a simple, tuned PID controller, our hybrid model successfully combines all three characteristics: it achieves the optimal Performance and Efficiency of the best DRL agent with the Reliability of a classical controller. This work demonstrates a practical methodology for using DRL to augment, rather than replace, trusted industrial control systems.

0.47

Efficient Scheduling of Discrete Industrial Processes Through Continuous Modeling

eess.SY ·2026-08-27 ·Ruike Lyu, Xiangbo Su, Ershun Du, Hongye Guo, Qixin Chen +1 more
top venuehighly cited similarity 0.65
Abstract · Published in: IEEE Transactions on Smart Grid ( Volume: 16, Issue: 6, November 2025)

The resource-task network (RTN) model has been widely applied to represent the technical constraints of complex industrial processes (IPs) such as steel-making, providing the basis for industrial demand response. However, the legacy RTN model contains numerous binary variables and applies different formulations for non-flexible and flexible processes, restricting its computational efficiency and applicability. To systematically improve the computational performance of IP models, we propose continuous RTN model (cRTN), a novel modeling approach that uses continuous variables to represent production tasks and progresses, which are then integrated into unified as well as computationally favorable formulations for the technical constraints in discrete IPs, including resource balance, task execution, waiting time limits, and production targets. Compared to the legacy models, cRTN features fewer binary variables, shorter solving time, and better scalability while maintaining the same accuracy. Numerical tests based on a steel plant demonstrate that cRTN is in typical cases 10 times faster than legacy models and remains tractable with increasing batch sizes, which in legacy models leads to larger problem scales and infeasible solving time. cRTN also achieves a reduction in energy costs by resolving the issue of rounding errors reported in legacy models.

0.65

Two-Timescale Reinforcement Learning for Real-Time Optimization and Economic NMPC: Experimental Validation

eess.SY ·2026-09-28 ·Saket Adhau, Jose Matias, Sebastien Gros, Sigurd Skogestad
similarity 0.64
Abstract · 12 pages, submitted to Journal of Process Control

We propose a reinforcement learning (RL) framework that tunes both Real-Time Optimization (RTO) and Economic Nonlinear Model Predictive Control (ENMPC) to address plant--model mismatch in process systems. Drawing on modifier-adaptation concepts, the method parameterizes the dynamic model, stage costs, constraints, and RTO modifiers, and uses Q-learning to adjust these parameters at two timescales: a fast update for the ENMPC layer and a slow update for the RTO layer. The framework is experimentally validated on a laboratory rig emulating a three-well subsea oil-production network. Using plant measurement data, the proposed RTO-RLMPC scheme achieves 8.6% higher economic profit than nominal ENMPC, preserves input feasibility of the deployed ENMPC policy and empirically satisfies the path constraints under disturbances and measurement noise, and drives the learned model parameters toward reference plant values, independently identified from experimental data, along the directions that most affect the economic objective. This work provides one of the first experimental demonstrations of RL-tuned ENMPC integrated with RTO.

0.24

Autonomous Model Lifecycle Management for Digital Twin-Based Manufacturing Control

cs.LG ·2026-09-19 ·Zhengyang, Gu, Thomas Cook, Fredaljohn Rohrbaugh, Joseph E. Hernandez +1 more
published similarity 0.64
Abstract · The paper has been accepted for publication in the 2026 IEEE International Conference on Systems, Man, and Cybernetics (SMC 2026). The final…

Manufacturing AI systems must autonomously adapt to continuous distributional shift from raw-material variability, ambient changes, and equipment aging, under strict safeguard and operator-trust requirements where model failures risk physical damage. This paper presents a closed-loop Cyber-Physical System (CPS) for autonomous model lifecycle management in automotive manufacturing, deployed since 2023. The system manages product-specialized model pairs: a sequence-to-sequence physics model (LPP) serving as a digital twin, and a deep Reinforcement Learning (RL) control policy (LCP) trained against it. Per retraining cycle, multiple model variants spanning architecture families and RL algorithms compete; only the best-scoring candidate advances. A Conductor orchestrator autonomously manages plant-wide model inventories with dependency-aware retraining and Proportional-Integral-Derivative (PID) fallback. Reflecting the principle of Human-Centric Intelligence, the LCP composite score embeds an operator-trust gate penalizing policies deviating from established practice; without it, 23% of policies are rejected by operators despite passing accuracy thresholds. Across multiple facilities, LCP-controlled processes achieve process stability improvements of 28-45% over uncontrolled baselines with zero safety incidents.

0.22

Data-Driven Resource Aggregation for Scalable Digital Twin and Simulation Model Generation

cs.LG ·2026-10-16 ·Yucheng Wu, Alp Akcay
similarity 0.63
Abstract

As industrial digitization accelerates, the use of digital twins has become widespread. However, modern manufacturing systems are highly dynamic and frequently reconfigured, making manual model development and updates increasingly difficult. Recent research on automated simulation model generation shows that historical data such as event logs can yield simulation models automatically. However, modeling every resource at full granularity leads to unnecessary complexity and computational cost. We propose a method for automatically selecting and aggregating resources to simplify simulation modeling. Our method uses ideas from inverse reinforcement learning to learn behavioral value representations of resources from production event logs. This process identifies and aggregates resources with similar functional contributions. Experiments on manufacturing systems, using throughput and system time as performance measures, demonstrate that value-based resource aggregation performs well when the goal is to substantially reduce complexity while preserving performance accuracy.

0.24