  Citation: Zaman, K.; Sun, Z.; Shah, S.M.; Shoaib, M.; Pei, L.; Hussain, A. Driver Emotions Recognition Based on Improved Faster R-CNN and Neural Architectural Search Network. Symmetry 2022, 14, 687. https:// doi.org/10.3390/sym14040687 Academic Editors: Gianluca Vinti and Sergei D. Odintsov Received: 7 February 2022 Accepted: 22 March 2022 Published: 26 March 2022 Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affil- iations. Copyright: © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). symmetry S S Article Driver Emotions Recognition Based on Improved Faster R-CNN and Neural Architectural Search Network Khalid Zaman 1 , Zhaoyun Sun 1, *, Sayyed Mudassar Shah 1 , Muhammad Shoaib 2 , Lili Pei 1 and Altaf Hussain 3 1 Information Engineering School, Chang’an University, Xi’an 710061, China; khalidzaman@chd.edu.cn (K.Z.); mudassarshah@chd.edu.cn (S.M.S.); peilili@chd.edu.cn (L.P.) 2 Department of Computer Science and IT, CECOS University, Peshawar 25000, Pakistan; mshoaib@cecos.edu.pk 3 Institute of Computer Science and IT, The University of Agriculture, Peshawar 25000, Pakistan; altafscholar@aup.edu.pk * Correspondence: chysun@chd.edu.cn; Tel.: +86-13572190029 Abstract: It is critical for intelligent vehicles to be capable of monitoring the health and well-being of the drivers they transport on a continuous basis. This is especially true in the case of autonomous vehicles. To address the issue, an automatic system is developed for driver’s real emotion recognizer (DRER) using deep learning. The emotional values of drivers in indoor vehicles are symmetrically mapped to image design in order to investigate the characteristics of abstract expressions, expression design principles, and an experimental evaluation is conducted based on existing research on the design of driver facial expressions for intelligent products. By substituting a custom-created CNN features learning block with the base 11 layers CNN model in this paper for the development of an improved faster R-CNN face detector that detects the driver’s face at a high frame per second (FPS). Transfer learning is performed in the NasNet large CNN model in order to recognize the driver’s various emotions. Additionally, a custom driver emotion recognition image dataset is being developed as part of this research task. The proposed model, which is a combination of an improved faster R-CNN and transfer learning in NasNet-Large CNN architecture for DER based on facial images, enables greater accuracy than previously possible for DER based on facial images. The proposed model outperforms some recently updated state-of-the-art techniques in terms of accuracy. The proposed model achieved the following accuracy on various benchmark datasets: JAFFE 98.48%, CK+ 99.73%, FER-2013 99.95%, AffectNet 95.28%, and 99.15% on a custom-developed dataset. Keywords: driver emotions recognition; computer vision; facial expression recognition; facial image symmetry; improved faster R-CNN; neural architecture search network 1. Introduction Drivers’ emotional states can impact their driving ability while driving a vehicle [1,2]. Due to the increasing sophistication of vehicles, recognizing the emotions of their drivers becomes more and more important. To ensure a more secure and pleasant ride, good infotainment can precisely detect the driver’s emotional state before making adjustments to the vehicle’s dynamics. In intelligent cars, it is critical to recognize the emotions of the driver because the vehicle can make decisions about what to do in certain situations based on the driver’s psychological state (for example driving modes, mood-altering songs, and autonomous driving). Facial expressions (FEs) are considered necessary in human- machine interfaces because they aid in expressing human emotions and feelings, which is essential in developing artificial intelligence. A new research area called facial expression recognition (FER) has been established. Recent years have seen significant advancements in deep learning-based image recognition techniques [3–8], and deep learning is becoming increasingly popular for FER. Although a person’s facial expressions often accurately reflect their genuine emotions, various factors can influence how accurately they do so. Symmetry 2022, 14, 687. https://doi.org/10.3390/sym14040687 https://www.mdpi.com/journal/symmetry