Citation: Fang, X.; Péter, T.; Tettamanti, T.Variable Speed Limit Control for the Motorway–Urban Merging Bottlenecks Using Multi-Agent Reinforcement Learning. Sustainability 2023, 15, 11464. https://doi.org/10.3390/ su151411464 Academic Editor: Giulio Erberto Cantarella Received: 31 May 2023 Revised: 17 July 2023 Accepted: 19 July 2023 Published: 24 July 2023 Copyright: © 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). sustainability Article Variable Speed Limit Control for the Motorway–Urban Merging Bottlenecks Using Multi-Agent Reinforcement Learning Xuan Fang * , Tamás Péter and Tamás Tettamanti Department of Control for Transportation and Vehicle Systems, Faculty of Transportation Engineering and Vehicle Engineering, Budapest University of Technology and Economics, M˝ uegyetem rkp. 3, H-1111 Budapest, Hungary; peter.tamas@kjk.bme.hu (T.P.); tettamanti.tamas@kjk.bme.hu (T.T.) * Correspondence: fangxuan@edu.bme.hu Abstract: Traffic congestion is a typical phenomenon when motorways meet urban road networks. At this special location, the weaving area is a recurrent traffic bottleneck. Numerous research activities have been conducted to improve traffic efficiency and sustainability at bottleneck areas. Variable speed limit control (VSL) is one of the effective control strategies. The primary objective of this paper is twofold. On the one hand, turbulent traffic flow is to be smoothed on the special weaving area of motorways and urban roads using VSL control. On the other hand, another control method is provided to tackle the carbon dioxide emission problem over the network. For both control methods, a multi-agent reinforcement learning algorithm is used (MAPPO: multi-agent proximal policy optimization). The VSL control framework utilizes the real-time traffic state and the speed limit value in the last control step as the input of the optimization algorithm. Two reward functions are constructed to guide the algorithm to output the value of the dynamic speed limit enforced within the VSL control area. The effectiveness of the proposed control framework is verified via microscopic traffic simulation using simulation of urban mobility (SUMO). The results show that the proposed control method could shape a more homogeneous traffic flow, and reduces the total waiting time over the network by 15.8%. In the case of the carbon dioxide minimization strategy, the carbon dioxide emission can be reduced by 10.79% in the recurrent bottleneck area caused by the transition from motorways to urban roads. Keywords: variable speed limit; reinforcement learning; multi-agent proximal policy optimization; road traffic control; traffic emission 1. Introduction As an important transportation infrastructure, motorways represent the overall level of a country’s transportation system to a large extent and play a vital role in the development of the national economy. However, motorways are alsofacing increasingly frequent traffic congestion [1]. As a road section that is prone to generating and spreading congestion, the traffic bottleneck area is an important research object to improve motorway traffic management capabilities. According to the specific time and location of occurrence, traffic bottleneck areas can be classified into recurring and nonrecurring bottleneck areas [2]. Recurring traffic bottleneck areas are the confluence area, diversion area, and weaving area formed when the main line of the motorway merges with the entrance ramp and exit ramp. Congestion frequently occurs in recurring traffic bottleneck areas [3]. The closure of lanes due to road construction, bad weather, or traffic accidents has stochastic characteristics, which are the causes of nonrecurring bottleneck areas [4–6]. Compared with motorways, the distribution of entrance and exit ramps at the merging area of motorways and urban roads is more concentrated, and adjacent entrances and exits are more closely connected with urban roads, resulting in more traffic conflicts in the merging area of motorways and urban roads [7,8]. Compared with urban roads, the Sustainability 2023, 15, 11464. https://doi.org/10.3390/su151411464 https://www.mdpi.com/journal/sustainability