The neural network, along with the computer vision system, determines in real time whether the car driver has fallen asleep or not. The use of this technology will lead to a significant increase in driving safety. The source code is available for everyone.
To train the neural network, the scientists used two datasets containing photos of faces with opened and closed eyes. These images were cropped so that only the eyes remained (Fig. 1). Subsequently, the script performing this operation was used in real time to crop webcam images. The final dataset contained more than 30,000 photos.
Within the framework of the task to be solved, it is more important to detect the state of sleep than the state of wakefulness. Therefore, the most appropriate metric is the recall metric, since the higher this metric is, the fewer sleeping drivers will be mistakenly recognized by the model as awake. Thus, the neural network will be less likely to produce false-negative results. However, the class of awake drivers significantly exceeds the class of sleeping drivers, and for this reason, the F1 score was chosen to take this circumstance into account. Otherwise, the model would always predict the state of sleep and would be unstable.
The neural network architecture is shown in Fig. 2. It consists of three parts: convolutional layers, flatten, and dense layers. Since the model solves the binary classification problem, a sigmoid activation function is used on the output layer. The output of the network is a value of 0 or 1, corresponding to the open or closed eyes in the image, respectively. After training, the model is used for real-time analysis of images from a webcam recording the face of the car driver. If the driver is asleep, the app interface displays a warning and activates an audio alarm. The criterion for the state of sleep that allows to distinguish it from a single blink is the result of recognizing closed eyes on 6 frames in a row, which corresponds to a time of about one second. The key advantage of the network is its extremely fast performance: it takes less than one millisecond to determine the driver’s status from a single frame. The source code is available for everyone.
Sleeping while driving leads to 91,000 accidents every year, of which about 800 are fatal. The use of the neural network will be useful for car and insurance companies interested in improving driving safety.