Deep Double Q-Learning Method for CAVs Traffic Signal Control 2020-01-5145

Urban intersection is the key element to determine the traffic operation of road network. Under the CAVs environment, the roadside control equipment of intersection can communicate with CAVs in real time, collect vehicle state data and optimize traffic control schemes. This paper presents a method for intersection traffic signal control based on deep learning of CAVs data. In addition, intelligent control agent of traffic signal (ICATS) is designed to simulate CAVs. ICATS can perceive real-time changes of traffic flow, model different conditions of intersection and generate the corresponding traffic signal scheme. ICATS used double Q-learning method combination with deep neural network, which is an effective model-independent deep learning algorithm. Moreover, the real traffic data is collected and tested in this paper for evaluating the experiment performance, including vehicle delay, number of passing vehicles, total stop times and passing time. ICATS is compared with another three popular traffic signal control algorithms, like static, actuated and delay_based control algorithm. By means of experimental analysis of real high-flow and low-flow data, the results indicate that signal timing efficiency can be significantly improved by using ICATS method. The experimental results demonstrate that the vehicle delay is shorter, and number of passing vehicles is larger compared with other methods. In this model, the optimal behavior strategy is found depending on trial and error, and the associated state-behavior value is updated by feedback. Finally, by means of this research, ICATS method can more accurately capture the dynamic correlation characteristics of traffic flow and signal timing, and obviously improve the operation efficiency of the intersection.