Effect of Delay on Dynamic Targets Tracking Performance and Behavior in Virtual Environment Vittorio Lippi, Carlo Alberto Avizzano, Denis Mottet, Emanuele Ruffaldi Abstract— We analyze the impact of the delay between the sensors and the visualization on the performance at a catching task in virtual environment. Testers have been asked to interact with a virtual environment displayed on a 2D screen in front of them, catching, with the virtual hand, virtual balls falling from the top of the screen. Two thresholds have been inferred: a behavioral threshold delay beyond which the behavioral organization is sensibly affected and a performance threshold delay beyond which task performance begins to worsen. We found a drop in performance when the total delay is over 70- 80 ms, yet without significant changes in the organization of successful catches. Although the presence of a threshold on behavior is still to be further investigated, these two thresholds give a practical indication for the design and the validation of virtual reality based training systems. This is especially important in the context of training human skills in virtual environment: in particular to assess the quality of a training platform simulating the three ball cascade juggling pattern. I. INTRODUCTION Virtual environments could be powerful training platform for real tasks. The training simulators not only can simulate the virtual environment of the training situation, but this environment can be enhanced via multimodal renderings, fostering the training process and accelerating the training effect [2]. In a simulated environment is possible to create the optimal conditions and the situations relevant to the task to be learned, to analyze variables not directly acces- sible in the real world, and to apply learning accelerator methods not possible real world physics, such as slowing down the simulation or giving visual hint [11]. On the other hand the differences between simulated and real world could lead to different behavioral schemes to achieve the same results and this can affect the quality of the training system. In particular, it is impossible to avoid some delay between sensors readings and visualization and this delay might impede performance. It is known that adding delay in visual feedback of one’s movement decreases performance in pointing [13] or tracking task [18], even though humans can adapt their control and use anticipation strategies to keep performance at its best [21]. Anticipation allows to adapt to the delay so to minimize errors, but some changes in more subtle aspects of the behavior are often observed [7]. For example, when bouncing a ball on racquet in VR, the delay is not consciously perceived below 90-100ms [16], though This work was supported by EU SKILLS Integrated Project Vittorio Lippi, Emanuele Ruffaldi and Carlo Alberto Avizzano are with PERCeptual RObotics Laboratory, Sant’Anna Superior School of Advanced Studies, Pisa. v.lippi@sssup.it e.ruffaldi@sssup.it carlo@sssup.it Denis Mottet is with Movement to health (EA2991), University Mont- pellier 1, France denis.mottet@univ-montp1.fr Fig. 1. System usage: the user stands close to a large screen controlling a virtual hand. The task consist in catching balls falling at a constant speed. behavioral changes in the phasing of movement occur for smaller delays [17]. Such behavioral changes can take the form of complex oscillatory behaviors induced by the time delay [1] or even delay-induced phase transitions leading to qualitatively different perception-action behaviors [20]. Though very brief, this review of previous works suggest that the effect of increasing the delay could be governed by two thresholds: • a behavioral threshold: increasing the delay has no effect up to this first threshold. If the delay is increased, behavior is successfully adapted to sustain performance level (e.g., using simple Smith predictor [15] or by integration of the delay in a dynamic synchronization process [19]). • a performance threshold: increasing the delay has no significant effect on performance up to this second threshold, but the behavior is differently organized [13], [18], [5], [17]. If the delay is increased, the performance starts to drop, because behavioral adaptability limits are attained. The presence of the performance threshold is rather obvious and it became a well documented issue that is critical for the design of VR and teleoperation systems [5]. Conversely, the presence of the behavioral threshold is often supposed but