Foreseeable Misuse in Automated Driving Vehicles - The Human Factor in Fatal Accidents of Complex Automation 2017-01-0059

Today, highly automated driving is paving the road for full autonomy. Highly automated vehicles can monitor the environment and make decisions more accurately and faster than humans to create safer driving conditions while ultimately achieving full automation to relieve the driver completely from participating in driving. As much as this transition from advanced driving assistance systems to fully automated driving will create frontiers for re-designing the in-vehicle experience for customers, it will continue to pose significant challenges for the industry as it did in the past and does so today. As we transfer more responsibility, functionality and control from human to machine, technologies become more complex, less transparent and making constant safe-guarding a challenge. With automation, potential misuse and insufficient system safety design are important factors that can cause fatal accidents, such as in TESLA autopilot incident. This paper investigates the human factor in fatal accidents from foreseeable misuse and insufficient system safety design perspectives. Fatal incidents, caused by different generations of autonomous systems are analyzed. Liability and product compliance topics of safety critical systems, procedures and controlled environment, are investigated for US regulations. Competency of automotive industry standards are evaluated within today’s automation needs, and compared against proven-in-use standards from other industries, especially from aviation. As a result, this paper unveils industry challenges in complex automated systems, caused by human factor, foreseeable misuse and insufficient system safety design to prevent fatal incidents.