Continuous Biomedical Monitoring in VR Scenarios of Socially Smart and Safe Autonomous Vehicle Interaction 1 st Tomasz Kocejko Gdansk University of Technology Gdansk, Poland tomasz.kocejko@pg.edu.pl 2 nd Abdeljalil Abbas-Turki University of Bourgogne Franche-Comte Belfort, France abdeljalil.abbas-turki@utbm.fr 3 nd Alexandre Brunoud University of Bourgogne Franche-Comte Belfort, France alexandre.brunoud@utbm.fr Abstract—Pedestrians, as vulnerable road users, pose safety challenges for autonomous vehicles (AVs). Their behavior, often unpredictable and subject to change, complicates AV-pedestrian interactions. To address this uncertainty, AVs can enhance safety by communicating their planned trajectories to pedestrians. In this research, we explore the interaction between pedestrians and autonomous vehicles within an industrial environment, focusing on how communicative behavior from the vehicles influences pedestrians’ physiology. We investigate the possibility of mea- suring biosignals while participants wear a VR headset and experiment a pedestrian crossing. Our preliminary study reveals subtle variations in delta rhythms when users immersed in VR simulations interact with AVs that either provide or withhold additional information. Index Terms—autonomous vehicles, biosignals, men machine interaction, EEG, VR I. I NTRODUCTION In conventional urban settings, pedestrian interactions with vehicles rely heavily on nonverbal communication, such as eye contact or gestures, to navigate safely across streets. However, the emergence of autonomous vehicles (AVs) in- troduces some challenges in this communication dynamic. Without a human driver present, AVs lack the means to convey intentions through facial expressions or gestures. Instead, they rely solely on turn signals and vehicle movements, leaving pedestrians uncertain about whether it is safe to cross. Recent research has delved into this issue, revealing that pedestrians feel notably less secure when encountering AVs lacking clear communication interfaces compared to traditional vehicles or AVs equipped with such interfaces [1]. The field of exter- nal Human Machine Interface (eHMI) applied to pedestrian- AV interaction focuses on optimal communication modalities between vehicles and pedestrians [12], [13], [15]. Numerous solutions have been suggested such as light strips, yet the two most efficient forms of communication include the speed curve, serving as an implicit communication method, and the light signal, facilitating direct communication. A study has This work was partially founded by Statutory Founds of Electronic, Telecommunication and Informatics Department of Gdansk University of Technology shown the effectiveness of such communication for managing a multi-agent environment [14]. These findings underscore the importance of effective communication systems in enhancing pedestrian trust and interaction with AVs. Moreover, studies emphasize that conveying vehicle intentions is paramount, regardless of factors like time of day, traffic density, or pedes- trian volume [2]. Efforts to develop effective communication interfaces, such as the Intent Communication System (ICS), aim to bolster pedestrian trust in AVs [3]. Studies confirm that the younger population has more trust in coexisting with AVs but the presence of the traffic lights has a pivotal role for them [4], [5]. Furthermore, investigations into the optimal interface design and interaction timing contribute to this evolving field [6]–[8]. Highly automated driving in commercial vehicles can in- crease road safety, reduce stress, and improve efficiency, economy, and ecological sustainability, while also enhancing the working environment for truck drivers. [9] Autonomous vehicles are more and more present in in- dustrial environments reducing manual work and improving efficiency [10]. Beyond urban settings, the integration of highly automated driving systems in commercial vehicles and industrial environments promises numerous benefits, including enhanced road safety, reduced stress, and improved efficiency. Consequently, AVs are becoming increasingly prevalent in industrial settings, streamlining manual tasks and bolstering overall efficiency. Despite the wealth of research on human- vehicle interactions in the context of AVs, there remains a gap in understanding the nuanced changes in individuals’ mental states when interacting with and coexisting alongside AVs, particularly in industrial environments. Most of the studies investigate human behavior in coop- eration with autonomous vehicles. We are wondering what changes in mental state co-working with AVs has. Moreover, we are wondering if biophysiology of a worker or pedestrian interacting as well as co-habiting with VAs is different from the one with regular cars where we rely on simple human-like communication (eye contact and gestures). Understanding the intricate dynamics of human-vehicle interaction is paramount 2024 16th International Conference on Human System Interaction (HSI) | 979-8-3503-6291-6/24/$31.00 ©2024 IEEE | DOI: 10.1109/HSI61632.2024.10613599 Authorized licensed use limited to: UT campus at Belfort Montebellard. Downloaded on August 19,2024 at 17:35:08 UTC from IEEE Xplore. Restrictions apply.