Learning to use automation: Behavioral changes in interaction with automated driving systems Yannick Forster a,b,⇑ , Sebastian Hergeth a , Frederik Naujoks a , Matthias Beggiato b , Josef F. Krems b , Andreas Keinath a a BMW Group, Knorrstr. 147, Munich 80937, Germany b Chemnitz University of Technology, Wilhelm-Raabe-Str. 43, 09120 Chemnitz, Germany article info Article history: Received 22 October 2018 Received in revised form 9 February 2019 Accepted 17 February 2019 Keywords: Automated driving Power-law of learning Human-machine interface Behavioral measures Method development abstract To evaluate human-machine interfaces for automated driving systems, a robust methodol- ogy is indispensable. The present driving simulator study investigated the effect of practice on behavioral measures (i.e., experimenter rating, reaction times, error rate) and the development of the preference-performance relationship for automated driving human- machine interfaces. In a within-subject design, N = 55 participants completed several transitions between manual, Level 2 and Level 3 automated driving. Behavioral measures followed the power law of practice with exception of transitions to manual and error rates for Level 3 automation. After the first block of interactions, preference no longer predicted performance. The preference-performance relationship remained stable after the second block of interactions, which is mainly due to a stabilization in behavioral parameters. To get a deeper insight into the evaluation of human-machine interfaces for automated driving, the results suggest the application of multi-method approaches. Furthermore, we found evidence for the influence of initial interactions for self-reported usability. Ó 2019 Elsevier Ltd. All rights reserved. 1. Introduction The quantum leap from Level 2 (L2) to Level 3 (L3) automated driving systems (ADS) availability on the commercial market is approaching quickly. L3 ADS are characterized by taking over longitudinal and lateral vehicle control. The driver does not have to constantly monitor correct system functioning (SAE, 2018). Instead, he/she may engage in non-driving related tasks (NDRT) such as watching a video on a tablet or reading a newspaper. However, it is the human driver’s respon- sibility to be readily available as a fallback performer in case the system function exceeds its operational design domain (ODD). The potential benefits of automated driving such as increased traffic and time efficiency, safety and comfort can only arise if market introduction is successful. To reach this aim, robust methods for the evaluation of both the system functions and the human-machine interface (HMI) are indispensable. The present study has two objectives. First, it focuses on the evaluation of different behavioral measures in the context of HMIs for automated driving with regard to effects of practice. https://doi.org/10.1016/j.trf.2019.02.013 1369-8478/Ó 2019 Elsevier Ltd. All rights reserved. ⇑ Corresponding author at: BMW Group, Knorrstr. 147, Munich 80937, Germany. E-mail addresses: yannick.forster@bmw.de (Y. Forster), sebastian.hergeth@bmw.de (S. Hergeth), frederik.naujoks@bmw.de (F. Naujoks), matthias. beggiato@psychologie.tu-chemnitz.de (M. Beggiato), josef.krems@psychologie.tu-chemnitz.de (J.F. Krems), andreas.keinath@bmw.de (A. Keinath). Transportation Research Part F 62 (2019) 599–614 Contents lists available at ScienceDirect Transportation Research Part F journal homepage: www.elsevier.com/locate/trf