Deep Reinforcement Learning-Based Path-Following Control for Underactuated Autonomous Underwater Vehicles

Xin Pan, Lin Huang, Liangjin Li, Song Wang

Autonomous Underwater Vehicles (AUVs) face significant challenges in path-following control due to strong environmental disturbances and model uncertainties. To address these issues, this paper proposes a model-free deep reinforcement learning framework, named ILLT (Improved LOS-LSTM-TD3), which integrates an integral line-of-sight (LOS) guidance law with the twin delayed deep deterministic policy gradient (TD3) algorithm. The framework treats the LOS look-ahead distance as a learnable optimization variable and incorporates an LSTM network to capture temporal motion dependencies. A progressive unfreezing transfer learning strategy, combined with attention-based feature–current fusion, is designed to enhance domain adaptation under varying ocean currents. Simulation results demonstrate that ILLT reduces the average cross-track error by 48.5% compared to the baseline ILT algorithm and by 66.4% compared to traditional PID control, while achieving significantly faster convergence in target domains. Physical experiments in tank and lake environments further validate the algorithm’s feasibility and robustness, with tracking errors approaching simulation results under moderate current conditions. These findings confirm the effectiveness of the proposed framework for underactuated AUV path-following tasks.