Multi-Component Joint Maintenance Decision for Electro-Hydraulic Servo Fatigue Testing Machine Based on Multi-Head Deep Reinforcement Learning

Peng Liu, Guotai Huang, Jialu Xi, Jiaqi Wu

To address the challenge of maintenance decision-making for critical components in electro-hydraulic servo material fatigue testing machine, characterized by weak state observability and difficulty in degradation prediction, a multi-component joint maintenance decision-making method based on multi-head deep reinforcement learning is proposed. Considering the heterogeneity of the degradation mechanisms and observation methods for the four components—bearing beam, fixture, main machine sensors, and hydraulic oil tank—a continuous-discrete hybrid state Markov decision process (HS-MDP) is constructed. To account for differences in maintenance strategies across components, a differentiated discrete action space for each component is designed, and engineering feasibility constraints are explicitly integrated into the policy through action masking. A data-quality loss term, determined by the degradation level of the sensors, is introduced into the reward function to align the optimization objective with the metrological properties of the fatigue testing machine. Based on the Branching Dueling DQN framework, a Q-network structure is constructed, incorporating a shared encoder, an inter-component attention mechanism, and multi-head branched outputs. Taking a 100 kN electro-hydraulic servo fatigue testing machine as a case study, comparisons with baseline strategies such as periodic maintenance, threshold-based condition-based maintenance (CBM), independent DQN, and PPO indicate that the proposed method reduces the average annual total cost by 60.3% compared to periodic maintenance and by 42.6% compared to threshold-based CBM. The number of failures decreases from 9.8 times/year to 1.4 times/year, while data efficiency increases from 82.1% to 96.2%. Ablation experiments and robustness tests further verify the critical contributions of three key design elements: action masking, inter-component attention, and data-quality loss.