  Citation: Ye, Q.; Feng, Y.; Candela, E.; Escribano Macias, J.; Stettler, M.; Angeloudis, P. Spatial-Temporal Flows-Adaptive Street Layout Control Using Reinforcement Learning. Sustainability 2022, 14, 107. https://doi.org/10.3390/su14010107 Academic Editors: Junfeng Jiao, Amin Azimian and Haizhong Wang Received: 29 November 2021 Accepted: 20 December 2021 Published: 23 December 2021 Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affil- iations. Copyright: © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). sustainability Article Spatial-Temporal Flows-Adaptive Street Layout Control Using Reinforcement Learning Qiming Ye * , Yuxiang Feng , Eduardo Candela , Jose Escribano Macias , Marc Stettler and Panagiotis Angeloudis Department of Civil and Environmental Engineering, Imperial College London, London SW7 2AZ, UK; y.feng19@imperial.ac.uk (Y.F.); e.candela-garza19@imperial.ac.uk(E.C.); jose.escribano-macias11@imperial.ac.uk (J.E.M.); m.stettler@imperial.ac.uk (M.S.); p.angeloudis@imperial.ac.uk (P.A.) * Correspondence: qiming.ye18@imperial.ac.uk Abstract: Complete streets scheme makes seminal contributions to securing the basic public right-of- way (ROW), improving road safety, and maintaining high traffic efficiency for all modes of commute. However, such a popular street design paradigm also faces endogenous pressures like the appeal to a more balanced ROW for non-vehicular users. In addition, the deployment of Autonomous Vehicle (AV) mobility is likely to challenge the conventional use of the street space as well as this scheme. Previous studies have invented automated control techniques for specific road management issues, such as traffic light control and lane management. Whereas models and algorithms that dynamically calibrate the ROW of road space corresponding to travel demands and place-making requirements still represent a research gap. This study proposes a novel optimal control method that decides the ROW of road space assigned to driveways and sidewalks in real-time. To solve this optimal control task, a reinforcement learning method is introduced that employs a microscopic traffic simulator, namely SUMO, as its environment. The model was trained for 150 episodes using a four-legged intersection and joint AVs-pedestrian travel demands of a day. Results evidenced the effectiveness of the model in both symmetric and asymmetric road settings. After being trained by 150 episodes, our proposed model significantly increased its comprehensive reward of both pedestrians and vehicular traffic efficiency and sidewalk ratio by 10.39%. Decisions on the balanced ROW are optimised as 90.16% of the edges decrease the driveways supply and raise sidewalk shares by approximately 9%. Moreover, during 18.22% of the tested time slots, a lane-width equivalent space is shifted from driveways to sidewalks, minimising the travel costs for both an AV fleet and pedestrians. Our study primarily contributes to the modelling architecture and algorithms concerning centralised and real-time ROW management. Prospective applications out of this method are likely to facilitate AV mobility-oriented road management and pedestrian-friendly street space design in the near future. Keywords: intelligent road infrastructure; Intelligent Transport System; reinforcement learning; Deep Deterministic Policy Gradient (DDPG); urban planning; street design; Autonomous Vehicles 1. Introduction The complete streets scheme is a mainstream engineering solution to improve road sharing for all road users [1,2]. It balances all users’ public right-of-way (ROW) and canalises road proportions according to respective travel demands [3,4]. A balanced ROW through the implementation of a complete street scheme could accommodate all modes of travel with rational road shares, an efficient operational environment and safe travel experiences [5,6]. Evidence shows that the complete streets scheme has considerably contributed to reducing road hazards, especially inter-modes traffic accidents, while maintaining rela- tively high transport efficiency [7,8]. However, their rigid and canalised thoroughfares Sustainability 2022, 14, 107. https://doi.org/10.3390/su14010107 https://www.mdpi.com/journal/sustainability