Research on Artificial Potential Field based Soft Actor-Critic Algorithm for Roundabout Driving Decision 2024-01-2871

Roundabouts are one of the most complex traffic environments in urban roads, and a key challenge for intelligent driving decision-making. Deep reinforcement learning, as an emerging solution for intelligent driving decisions, has the advantage of avoiding complex algorithm design and sustainable iteration. For the decision difficulty in roundabout scenarios, this paper proposes an artificial potential field based Soft Actor-Critic (APF-SAC) algorithm. Firstly, based on the Carla simulator and Gym framework, a reinforcement learning simulation system for roundabout driving is built. Secondly, to reduce reinforcement learning exploration difficulty, global path planning and path smoothing algorithms are designed to generate and optimize the path to guide the agent. Then, considering the complex interactions between vehicles in roundabouts, a Markov decision process model is constructed, and a coupled longitudinal and lateral action space, a vectorized state space based on roundabout scenarios, and a reward function based on artificial potential field are designed, and the APF-SAC algorithm is proposed. Finally, simulation experiments under different traffic densities show that compared to rule-based driving decisions, the deep reinforcement learning method can significantly improve decision safety and driving efficiency in roundabout scenarios, with the maximum safety improvement of 10.4% and the maximum driving efficiency improvement of 13.2%, demonstrating the superior performance of the APF-SAC algorithm for roundabout driving decisions. This research provides an effective approach for applying reinforcement learning algorithms to complex urban autonomous driving decisions.