A Deep Reinforcement Learning-Based Concurrency Control of Federated Digital Twin for Software-Defined Manufacturing Systems

Rubab Anwar, Jin-Woo Kwon, Won-Tae Kim

Modern manufacturing demands real-time, scalable coordination that legacy manufacturing management systems cannot provide. Digital transformation encompasses the entire manufacturing infrastructure, which can be represented by digital twins for facilitating efficient monitoring, prediction, and optimization of factory operations. A Federated Digital Twin (FDT) emerges by combining heterogeneous digital twins, enabling real-time collaboration, data sharing, and collective decision-making. However, deploying FDTs introduces new concurrency control challenges, such as priority inversion and synchronization failures, which can potentially cause process delays, missed deadlines, and reduced customer satisfaction. Traditional concurrency control approaches in the computing domain, due to their reliance on static priority assignments and centralized control, are inadequate for managing dynamic, real-time conflicts effectively in real production lines. To address these challenges, this study proposes a novel concurrency control framework combining Deep Reinforcement Learning with the Priority Ceiling Protocol. Using SimPy-based discrete-event simulations, which accurately model the asynchronous nature of FDT interactions, the proposed approach adaptively optimizes resource allocation and effectively mitigates priority inversion. The results demonstrate that against the rule-based PCP controller, our hybrid DRLCC enhances completion time maximum of 24.27% to a minimum of 1.51%, urgent-job delay maximum of 6.65% and a minimum of 2.18%, while preserving lower-priority inversions.