Browse Publications Technical Papers 2020-01-1211

Driver Physiological Drowsiness Behavior Detection and Analysis using Vision-based Multimodal Features for Driving Safety 2020-01-1211

Driving safety has always been a fundamental concern in transportation systems. Driving inattention caused by drowsiness has been a significant reason for vehicle crash accidents according to United States Traffic Safety Culture Index report, and there is an essential need to improve assistance driving safety by understanding the driver behaviors. Towards real-time drowsy driving monitoring, we propose an in-vehicle driver assistant system to monitor driver states for drowsiness behavior recognition and analysis. First, an infrared camera is deployed inside the vehicle to capture the driver’s facial and head information, in which scenarios, the driver is allowed to wear glasses or sunglasses during driving. Second, vision-based multimodal features, facial landmarks and head pose are extracted efficiently by the ensemble of regression trees based facial landmarks estimation method and a convolutional neural network (CNN) recognition model. Finally, an extreme learning machine (ELM) model is proposed to fuse the facial landmark, recognition model and pose orientation for drowsiness detection. The system gives promptly warning to the driver once a drowsiness event is detected. The proposed machine learning recognition model are trained in drowsy driving dataset and validated in both public dataset and field tests. Comparing to the end-to-end CNN recognition model, the proposed multimodal fusion with ELM detection model allows faster and more accurate detection with minimal intervention. The experimental result demonstrates that the proposed system is able to provide prompt and effective drowsy driving alerts under various light conditions to augment driving safety. The field test demo of this research is available at


Subscribers can view annotate, and download all of SAE's content. Learn More »


Attention: This item is not yet published. Pre-Order to be notified, via email, when it becomes available.
Members save up to 18% off list price.
Login to see discount.
Special Offer: With TechSelect, you decide what SAE Technical Papers you need, when you need them, and how much you want to pay.