On the Latency in Vehicular Control using Video Streaming over Wi-Fi Pratik Sharma 1 , Devam Awasare 1 , Bishal Jaiswal 1 , Srivats Mohan 1 , Abinaya N. 2 , Ishan Darwhekar 1 , Anand SVR 1 , Bharadwaj Amrutur 1,2 , Aditya Gopalan 1 , Parimal Parag 1 , Himanshu Tyagi 1 1 Department of Electrical Communication Engineering, Indian Institute of Science 2 Robert Bosch Centre for Cyber-Physical Systems, Indian Institute of Science {pratiksharma, devamawasare, bishalj, srivatsmohan, abinayan, ishand, anandsvr, amrutur, aditya, parimal, htyagi}@iisc.ac.in Abstract—We consider the use of Wi-Fi (IEEE 802.11n/r) network for remote control of a vehicle using video transmission on the uplink and control signals for the actuator on the downlink. We have setup a network with multiple access points (AP) providing indoor and outdoor coverage, which connects an unmanned ground vehicle (UGV) to a remote command center. Additionally, our setup includes a redundant IEEE 802.11p link for sending control messages over downlink with high reliability and low latency. We study the end-to-end communication de- lay and complete a latency profiling for each sub-component, including the video codec and the Wi-Fi links. Furthermore, we provide guidelines for practical design choices including the optimal configuration of the scanning process during handoffs and the codec parameters for delay optimization. Overall, our proposed configuration reduces the end-to-end delay significantly in comparison with the default configuration. I. I NTRODUCTION We have designed an experimental setup to profile the end-to-end latency for communication between a remotely driven UGV and its control center. Autonomous or remote control driving is possible through multimodal sensing using techniques such as RADAR (RAdio Detection And Ranging), LIDAR (Light Detection and Ranging), and video streams, where video consumes the largest portion of the uplink throughput. For simplicity, we have setup a vehicle with only video streaming as sensing mechanism for remote driving and have connected it to a command center over the Wi-Fi network. We present a systematic analysis of the end-to-end latency in the transmission of video stream over the uplink, including the video codec delays at both ends, and control command over the downlink. Based on our analysis, we have identified the key parameters that are relevant for delays and identify their best-case scenario values. In the remainder of this section, we briefly discuss our setup, the key contributors to latency, prior art, and our specific contributions. Each of these components is elaborated later. A. Components of our setup We consider a scenario where a mobile node with live video stream and limited computation power is connected over Wi- Fi to a command center with a powerful server. We consider remote driving by an operator and autonomous braking by detection at the command center. Such real-time controls over network require an uplink with high throughput & low latency Fig. 1: Block diagram for communication between UGV and command center. and a low latency downlink. In addition, edge computing capability is required at the mobile node to enable local decision making whenever needed, and Multi-access Edge Computing (MEC) capabilities in network elements to enable real-time network optimization. Keeping these requirements in mind, we have deployed a Wi-Fi network with multiple access points (APs) connected to the command center with powerful compute over fiber. Our Wi-Fi network includes both IEEE 802.11n and IEEE 802.11r, along with IEEE 802.11p. This testbed is live and will support further research and development beyond what is being reported. A block view of our end-to-end setup is depicted in Fig. 1. B. The main contributors to latency Low latency is a critical requirement for real-time control over networks. In spite of significant recent interest in low latency communication, it is unclear what is the latency possible using existing network deployments. In this work, focussing on a Wi-Fi deployment, we have identified three main contributors to latency: (1) The sampling, encoding, and decoding delays of the video codec, (2) the transmission delays for the network, and perhaps most importantly, (3) the delay caused by handoffs in Wi-Fi. Note that a single Wi-Fi AP can provide coverage of 50 - 100 m in outdoor deployments with foliage, whereby a mobile node will often encounter handoffs as it navigates using the network. Thus, the handoff delays dominate latency in Wi-Fi and must be carefully mitigated. In comparison, the transmission delays are negligible. In fact, the video codec delays much exceed the transmission delays and must be addressed. C. Prior work The paper [1] (see, also, [2]–[4]) studies a city-scale Wi-Fi deployment for vehicular communication and provides an ex-