PREPRINT Simultaneous Run-Time Measurement of Motion-to-Photon Latency and Latency Jitter Jan-Philipp Stauffert * University of W¨ urzburg Florian Niebling † University of W¨ urzburg Marc Erich Latoschik ‡ University of W¨ urzburg Motion to Photon Latency Figure 1: Illustration of the apparatus developed to measure the Motion-to-Photon latency. A microcontroller (middle) reads the difference between the rotation of a tracked controller as moved by a motor (left) and the reported rotation on the Vive display (right). The illustration shows the experimental prototype which disassembled the Head-Mounted Display for easier access to the internals, i.e., the lenses and displays. ABSTRACT Latency in Virtual Reality (VR) applications can have numerous detrimental effects, e.g., a hampered user experience, a reduced user performance, or the occurrence of cybersickness. In VR envi- ronments, latency usually is measured as Motion-to-Photon (MTP) latency and reported as a mean value. This mean is taken during some specific intervals of sample runs with the target system, of- ten detached in significant aspects from the final target scenario, to provide the necessary boundary conditions for the measurements. Additionally, the reported mean value is agnostic to dynamic and spiking latency behavior. This paper introduces an apparatus that is capable of determining per-frame MTP latency to capture dynamic MTP latency and latency jitter in addition to the commonly reported mean values of latency. The approach is evaluated by measuring MTP latency of a VR simulation based on the Unreal engine and the HTC Vive as a typical consumer-grade Head-Mounted Display (HMD). In contrast to previous approaches, the system does not rely on the HMD to be fixed to an external apparatus, can be used to assess any simulation setup, and can be extended to continuously measure latency during run-time. We evaluate the accuracy of our apparatus by injecting a controlled artificial latency in a VR simu- lation. We show that latency jitter artifacts already occur without system load, potentially caused by the tracking of the specific HMD, and how mean latency and jitter increase under system load, leading to dropped frames and an overall degraded system performance. The presented system can be used to monitor latency and latency jitter as critical simulation characteristics necessary to report and control to avoid unwanted effects and detrimental system performance. Index Terms: D.4.8 [Operating Systems]: Performance— * e-mail:jan-philipp.stauffert@uni-wuerzburg.de † e-mail:florian.niebling@uni-wuerzburg.de ‡ e-mail:marc.latoschik@uni-wuerzburg.de Measurements; H.5.1 [Information Interfaces and Presentation]: Multimedia Information Systems—Artificial, augmented, and vir- tual realities 1 I NTRODUCTION Each instruction of a computer system has an associated execution time. As a result, any output that is calculated after changing user input will always be delayed, introducing latency into the human- computer interaction loop. The impact this delay causes on the system qualities of usability and user experience depends on the interactivity and potential real-time requirements of the human- computer interface and the employed interaction metaphor [6]. Even users of a spreadsheet software in a 2D graphical user interface (GUI) expect a timely response to a mouse click. The response should feel to be instantaneous. The conditions to experience feedback as instantaneous depend on several factors of the interaction metaphor. Direct interaction metaphors are more sensitive to delays than indirect ones. Here, VR systems are specifically sensitive to delays between input and output processing since they directly and continu- ously couple input, including head movements, to the visual output. The overall latency in VR between an action, e.g, moving an input controller measuring hand or head movements, and its correspond- ing effect shown on a screen is denoted as Motion-to-Photon (MTP) latency. Unwanted delays during the input-to-output processing not only risk to annoy users but they potentially might induce more severe consequences of visually-induced motion sickness (VIMS) and cybersickness [25]. Hence, a central requirement of VR systems is to measure and finally control a VR system’s latency behavior to judge its performance before negative consequences arise. Deducing the latency from code inspection and counting the execution times of all instructions of a VR application is largely out of reach today. The interplay of all soft- and hardware sub-systems of modern general-purpose computer systems introduce a dynamic complexity that can only be observed as a black box [28]. Low- level system interrupts of the system’s motherboard, modern CPU speed-up approaches including parallelization, caching, and branch prediction, as well as high-level operating system and application- layer aspects of concurrency, multi-threading and scheduling, and