Safety Argument Considerations for Public Road Testing of Autonomous Vehicles 2019-01-0123
Autonomous vehicle (AV) developers test extensively on public roads, potentially putting other road users at increased risk. While increased use of simulation can help manage this risk, some amount of road testing of vehicles will be required before full scale deployment. This paper describes considerations for safety arguments involving human safety driver supervision of testing. Such arguments must include three main elements: (1) the human must be alert and engaged, (2) the human must have adequate time to react to system misbehavior, and (3) the vehicle must properly respond to human over-ride actions. There are a number of subtleties in such a safety argument. Human alertness can degrade quickly while supervising autonomy, and is unlikely to be perfect. This necessitates an analysis that considers both the ability of the human to focus on the safety task and the maturity of the autonomy technology to evaluate the probability of a coincident failure. Human reaction to an autonomy failure takes time, and potentially requires the human to mentally model expected AV behavior to minimize false alarm disengagements. This will likely result in a requirement that AV test platforms be more conservative than would be desirable in production operation, potentially leaving a performance gap due to validation constraints. While ADAS features might be counted upon to provide defense in depth to failures, the nature of their off-the-shelf validation arguments that assume a human driver is in control of the vehicle can prove problematic. And finally, vehicle disengagements can stress baseline vehicle functions in ways that likely differs from OEM underlying platform design assumptions. A credible safety argument for on-road AV testing must successfully address the risks presented by these various considerations, ensuring that the public is not exposed to undue risk while AV technology is being developed and validated. Similar issues seem likely to affect any safety argument for production vehicles with lower levels of autonomy that depend upon human supervision for safety.
Philip Koopman, Beth Osyk PhD
Carnegie Mellon University, Edge Case Research