1 Potential Game Based Decision-Making Frameworks for Autonomous Driving Mushuang Liu, Ilya Kolmanovsky, H. Eric Tseng, Suzhou Huang, Dimitar Filev, and Anouck Girard Abstract—Decision-making for autonomous driving is chal- lenging, considering the complex interactions among multiple traffic agents (e.g., autonomous vehicles (AVs), human drivers, and pedestrians) and the computational load needed to evaluate these interactions. This paper develops two general potential game based frameworks, namely, finite and continuous potential games, for decision-making in autonomous driving. The two frameworks account for the AVs’ two types of action spaces, i.e., finite and continuous action spaces, respectively. We show that the developed frameworks provide theoretical guarantees, including 1) existence of pure-strategy Nash equilibria, 2) convergence of the Nash equilibrium (NE) seeking algorithms, and 3) global optimality of the derived NE (in the sense that both self- and team- interests are optimized). In addition, we provide cost func- tion shaping approaches to constructing multi-agent potential games in autonomous driving. Moreover, two solution algorithms, including self-play dynamics (e.g., best response dynamics) and potential function optimization, are developed for each game. The developed frameworks are then applied to two different traffic scenarios, including intersection-crossing and lane-changing in highways. Statistical comparative studies, including 1) finite po- tential game vs. continuous potential game, and 2) best response dynamics vs. potential function optimization, are conducted to compare the performances of different solution algorithms. It is shown that both developed frameworks are practical (i.e., computationally efficient), reliable (i.e., resulting in satisfying driving performances in diverse scenarios and situations), and robust (i.e., resulting in satisfying driving performances against uncertain behaviors of the surrounding vehicles) for real-time decision-making in autonomous driving. I. I NTRODUCTION During the past five years, the autonomous vehicle (AV) market has attracted more than $50 billion in investment from major carmakers, tech giants and start-ups, and is expected to continue its growth by 36% in the next two years [1]. Although the prospect of AVs looks promising and bright, technical challenges, such as intelligent decision-making in diverse traffic scenarios, still remain to be addressed before the AVs can routinely drive on the roads [2]–[4]. Developing practical and reliable decision-making algorithms for AVs is challenging, due to the complexities caused by the interactions of multiple traffic participants and the requirement of real-time decisions [5]–[7]. To generate safe and effective decisions for the ego vehicle, the interactions with its surrounding traffic agents, includ- M. Liu, I. Kolmanovsky, and A. Girard, are with the Department of Aerospace Engineering, University of Michigan, Ann Arbor, MI, USA (email: mushuanl@umich.edu, ilya@umich.edu, and anouck@umich.edu). H. E. Tseng, S. Huang, and D. Filev are with Ford Research and In- novation Center, 2101 Village Road, Dearborn, MI 48124, USA (e-mail: htseng@ford.com, shuang10@ford.com, and dfilev@ford.com). This work is supported by Ford Motor Company. ing AVs, human-driven vehicles, and pedestrians, have to be considered [8]–[11]. As each traffic agent has its own objective (also called self-interest), the multi-agent decision- making problem is intrinsically a multi-player game problem [7], [12]–[15]. In a game-theoretic setting, each agent aims to optimize its self-interest, which is affected by not only its own actions but also the actions of its surrounding agents. Along these lines, reference [16] develops a two-player nonzero- sum game to model the interactions between a through ve- hicle and a merging vehicle in merging-giveaway scenarios. Papers [12], [13] consider a two-player Stackelberg game and a pairwise normal-form game, respectively, to address the decision-making in lane-changing scenarios. To capture the agents’ interactions in intersection-crossing scenarios, a leader-follower game and a normal-form game are introduced, respectively, in [17] and [14]. To solve for Nash equilibrium (NE) in multi-agent Markov games, a best response dynamics based solution algorithm is proposed in [18], with numerical examples in two-vehicle highway merging scenarios. Due to scalability issues, most of the aforementioned works focus on two-agent scenarios or handle multi-agent scenarios in a pairwise manner. However, as pointed out in [19], such pairwise treatment can be detrimental to the AVs’ driving performance, as it only captures the local interaction between a pair of vehicles and ignores the interaction correlations among multiple vehicle pairs. To address this challenge while main- taining computational efficiency, [19] proposes a hierarchical game framework with a game player selection procedure to reduce the number of game players while maintaining critical interaction information. Although multi-player games provide promising solutions to model the interactions of multiple traf- fic agents, theoretical guarantees in such multi-player games, including the existence of solution, convergence of solution algorithms, and global optimality of the derived solution, are still lacking. Potential games [20] are a special class of non-cooperative games where a real-valued global function (called potential function) exists such that a change of an agent’s self-interest by its own strategy deviation is equal to the change of the potential function. Potential games have appealing properties, including existence and attainability of pure-strategy Nash equilibria, and as such, attract increasing attention in eco- nomics [21], social systems [22], and wireless networks [23]. However, verifying whether a given game is a potential game is not easy [24], which limits the applications of potential games to a wider range of engineering problems, e.g., au- tonomous driving. This paper proposes a novel multi-player game theory based arXiv:2201.06157v1 [eess.SY] 16 Jan 2022