Reinforcement Learning-Based Metro Train Tracking Control to Overcome Input Time Delay

Abstract

Speed tracking and precise stop control of metro trains play a key role in train operation. Reinforcement learning (RL) is one of the methods that can solve train control challenges and adapt to various environments. However, when applying RL to train control, the input time delay of the train presents particular difficulties. This paper proposes a RL design method that integrates prediction techniques and the deep deterministic policy gradient algorithm to overcome the input time delay. The training and performance evaluation of the proposed RL is conducted using the validated Automatic Train Operation (ATO) simulator for Seoul Metro Line 5. The results demonstrate that prediction-based RL (PRL) controllers outperform RL without prediction controllers. Additionally, the PRL controller robustness is verified through performance comparisons with a classical model-based approach across various realistic scenarios.