Browse Publications Technical Papers 2020-01-1204

Modes of Automated Driving System Scenario Testing: Experience Report and Recommendations 2020-01-1204

With the widespread development of automated vehicles (AV), it is imperative that standardized testing methodologies be developed to ensure safety and functionality. One way to perform whole-system testing is to observe the behavior of a subject automated vehicle (SV) in many predefined scenarios. This paper presents a method for automated vehicle scenario testing and four modes for applying that method. The four modes are: closed-course testing with manually controlled actors, closed-course testing with automatically controlled actors, simulation testing, and closed-course testing with mixed reality. We discuss the advantages and disadvantages of each mode. We have excluded open-road testing from our discussion as it requires a different testing method from the one presented here. In collaboration between the Waterloo Intelligent Systems Engineering (WISE) Lab and AAA Northern California, Nevada & Utah, we executed six automated vehicle test scenarios involving pedestrians, vehicles, and road debris both at a closed course, in simulation, and in mixed reality. The SV was the University of Waterloo's automated vehicle, the "UW Moose". Drawing on both data and the experience gained from executing these test scenarios, we describe the advantages and disadvantages of scenario testing in simulation and at a closed course with manual and automated actors as well as in mixed reality. Next, we use an analytical approach to describe the advantages and disadvantages of mixed reality scenario testing and briefly discuss its implementation. Finally, we present guidelines for choosing between the four modes and discuss related work.


Subscribers can view annotate, and download all of SAE's content. Learn More »


Attention: This item is not yet published. Pre-Order to be notified, via email, when it becomes available.
Members save up to 18% off list price.
Login to see discount.
Special Offer: With TechSelect, you decide what SAE Technical Papers you need, when you need them, and how much you want to pay.