IEEE/CAA JOURNAL OF AUTOMATICA SINICA, VOL. 2, NO. 1, JANUARY 2015 11 A Predator-prey Particle Swarm Optimization Approach to Multiple UCAV Air Combat Modeled by Dynamic Game Theory Haibin Duan, Pei Li, and Yaxiang Yu Abstract—Dynamic game theory has received considerable attention as a promising technique for formulating control actions for agents in an extended complex enterprise that involves an adversary. At each decision making step, each side seeks the best scheme with the purpose of maximizing its own objective func- tion. In this paper, a game theoretic approach based on predator- prey particle swarm optimization (PP-PSO) is presented, and the dynamic task assignment problem for multiple unmanned combat aerial vehicles (UCAVs) in military operation is decomposed and modeled as a two-player game at each decision stage. The optimal assignment scheme of each stage is regarded as a mixed Nash equilibrium, which can be solved by using the PP-PSO. The effectiveness of our proposed methodology is verified by a typical example of an air military operation that involves two opposing forces: the attacking force Red Red Red and the defense force Blue Blue Blue. Index Terms—Unmanned combat aerial vehicle (UCAV), game theory, air combat, predator-prey, particle swarm optimization (PSO), Nash equilibrium. I. I NTRODUCTION C OMPARED to unmanned combat aerial vehicles (UCAVs) that perform solo missions, greater efficiency and operational capability can be realized from teams of UCAVs operating in a coordinated fashion [1−5] . Designing UCAVs with intelligent and coordinated action capabilities to achieve an overall objective is a major part of multiple UCAVs control in a complicated and uncertain environment [6−10] . Actually, a military air operation involving multiple UCAVs is a complex dynamic system with many interacting decision- making units which have even conflicting objectives. Modeling and control of such a system is an extremely challenging task, whose purpose is to seek a feasible and optimal scheme to assign the limited combat resource to specific units of the adversary while taking into account the adversary ′ s possible defense strategies [8, 11] . The difficulty lies not only in that it is often very difficult to mathematically describe the underlying Manuscript received July 24, 2013; accepted July 18, 2014. This work was supported by National Natural Science Foundation of China (61425008, 61333004, 61273054), Top-Notch Young Talents Program of China, and Aeronautical Foundation of China (2013585104). Recommended by Associate Editor Changyin Sun Citation: Haibin Duan, Pei Li, Yaxiang Yu.A predator-prey particle swarm optimization approach to multiple UCAV air combat modeled by dynamic game theory. IEEE/CAA Journal of Automatica Sinica, 2015, 2(1): 11-18 Haibin Duan, Pei Li, and Yaxiang Yu are with the Science and Technology on Aircraft Control Laboratory, School of Automation Sci- ence and Electrical Engineering, Beihang University (BUAA), Beijing 100191, China (e-mail: hbduan@buaa.edu.cn; peilibuaa@asee.buaa.edu.cn; yaxiangyu03@asee.buaa.edu.cn). processes and objectives of the decision maker but also in that the fitness of one decision maker depends on both its own control input and the opponent ′ s strategies as well. Dynamic game theory has received increasingly intensive attention as a promising technique for formulating action strategies for agents in such a complex situation, which in- volves competition against an adversary. The priority of game theory in solving control and decision-making problems with an adversary opponent has been shown in many studies [12−15] . A game theory approach was proposed for target tracking problems in sensor networks in [14], where the target is assumed to be an intelligent agent who is able to maximize filtering errors by escaping behavior. The pursuit-evasion game formulations were employed in [16] for the development of improved interceptor guidance laws. Cooperative game theory was used to ensure team cooperation by Semsar-Kazerooni et al. [13] , where a team of agents aimed to accomplish consensus over a common value for their output. Although finding the Nash equilibrium in a two-player game may be easy since the zero-sum version can be solved in polynomial time by linear programming, this problem has been proved to be indeed PPAD-complete [17−18] . So the problem of computing Nash equilibria in games is computationally extremely difficult, if not impossible. Based on the analogy of the swarm of birds and the school of fish, Kennedy and Eberhart developed a powerful optimization method, particle swarm optimization (PSO) [19−20] , addressing the social inter- action, rather than purely individual cognitive abilities. As one of the most representative method aiming at producing com- putational intelligence by simulating the collective behavior in nature, PSO has been seen as an attractive optimization tool for the advantages of simple implementation procedure, good performance and fast convergence speed. However, it has been shown that this method is easily trapped into local optima when coping with complicated problems, and various tweaks and adjustments have been made to the basic algorithm over the past decade [20−22] . To overcome the aforementioned problems, a hybrid predator-prey PSO (PP-PSO) was firstly proposed in [21] by introducing the predator-prey mechanism in the biological world to the optimization process. Recently, bio-inspired computation in UCAVs have attracted much attention [23−25] . However, the game theory and solutions to the problem of task assignment have been studied indepen- dently. The main contribution of this paper is the development