Browse Publications Technical Papers 2018-01-0124
2018-04-03

A Comprehensive Testing and Evaluation Approach for Autonomous Vehicles 2018-01-0124

Performance testing and evaluation always plays an important role in the developmental process of a vehicle, which also applies to autonomous vehicles. The complex nature of an autonomous vehicle from architecture to functionality demands even more quality-and-quantity controlled testing and evaluation than ever before. Most of the existing testing methodologies are task-or-scenario based and can only support single or partial functional testing. These approaches may be helpful at the initial stage of autonomous vehicle development. However, as the integrated autonomous system gets mature, these approaches fall short of supporting comprehensive performance evaluation. This paper proposes a novel hierarchical and systematic testing and evaluation approach to bridge the above-mentioned gap. In this paper, firstly a three-dimensional evaluation model conforming to the functional architecture of autonomous vehicles was built, with each dimension representing one of the three key functional layers of autonomous vehicle including sensing & perception, decision-making ﹠ planning, control & execution. Each dimension has a set of metrics carefully defined with their weights fairly determined based on an entropy weights method. Then, considering environment effect on vehicle functions, we innovatively determine task-scenarios for testing the performance of each dimension. Besides, we design a hierarchical systematic testing method which could specially testing each function layer of autonomous vehicle. Fuzzy comprehensive and TOPSIS evaluation method was proposed to quantitatively evaluate the comprehensive performance of autonomous vehicles under defined task-scenarios. Finally, our methods are used to evaluate three candidate vehicles based on simulation scenario in PanoSim. Compared to traditional approach based on external task performance, the proposed approach can not only provide convincing results of the overall system performance but can also peek into each of the key functional layer and provide insights about their performance. Therefore, this approach provides better guidance for autonomous vehicle research.

SAE MOBILUS

Subscribers can view annotate, and download all of SAE's content. Learn More »

Access SAE MOBILUS »

Members save up to 18% off list price.
Login to see discount.
Special Offer: Download multiple Technical Papers each year? TechSelect is a cost-effective subscription option to select and download 12-100 full-text Technical Papers per year. Find more information here.
We also recommend:
TECHNICAL PAPER

Study on Target Tracking Based on Vision and Radar Sensor Fusion

2018-01-0613

View Details

TECHNICAL PAPER

Localization and Perception for Control and Decision Making of a Low Speed Autonomous Shuttle in a Campus Pilot Deployment

2018-01-1182

View Details

JOURNAL ARTICLE

Developing a Standardized Performance Evaluation of Vehicles with Automated Driving Features

12-02-03-0011

View Details

X