Coalition Formation through Learning in Autonomic Networks Tao Jiang and John S. Baras Abstract— Autonomic networks rely on the cooperation of participating nodes for almost all their functions. However, due to resource constraints, nodes are generally selfish and try to maximize their own benefit when participating in the network. Therefore, it is important to study mechanisms, which can be used as incentives for cooperation inside the network. In this paper, the interactions among nodes are modelled as games. A node joins a coalition if it decides to cooperate with at least one node in the coalition. The dynamics of coalition formation proceed via nodes that interact strategically and adapt their behavior to the observed behavior of others. We present conditions that the coalition formed is stable in terms of Nash stability and the core of the coalitional game. I. I NTRODUCTION Autonomic networks rely on the cooperation of participat- ing nodes for almost all their functions, for instance, to route data between source and destination pairs that are outside each other’s communication range. However, because nodes are resource constrained, we deal with networks composed of selfish users who are trying to maximize their own benefit from participation in the network. In particular, we assume that each user is in complete control of his network node. In the routing example, the fundamental user decision is between forwarding or not forwarding data packets sent by other users. Given the constraints (mostly related to battery power) that the user faces, there is a very real cost incurred when choosing to forward. So, all users would like to send their own data packets, but not forward those of other users. Unfortunately, if all users were to do that, the network would collapse. In order to form the necessary infrastructure that makes multi-hop communications achievable, cooperation enforcement mechanisms are needed to cope with such selfish behavior of nodes in autonomic networks. The conflict between the benefit from cooperation and the required cost for cooperation naturally leads to game- theoretic studies, where each node strategically decides the degree to which it volunteers its resources for the common good of the network. The players in game theory attempt to maximize an objective function which takes the form of a payoff. Srinivasan et al. [1] address the problem of cooper- ation among energy constrained nodes and devised behavior This work is prepared through collaborative participation in the Commu- nications and Networks Consortium sponsored by the U.S. Army Research Laboratory under the Collaborative Technology Alliance Program, Coop- erative Agreement DAAD19-01-2-0011. Research is also supported by the U.S. Army Research Office under grant No DAAD19-01-1-0494. T. Jiang is with the Institute for Systems Research, University of Maryland, College Park, MD 20742 tjiang@umd.edu J. S. Baras is with the Institute for Systems Research, Department of Electrical and Computer Engineering, Department of Computer Science, The Fischell Department of Bioengineering, University of Maryland, Col- lege Park, MD 20742 baras@umd.edu strategies of nodes that constitute a Nash equilibrium. In [2], there is a link between two nodes if they agree to cooperate. These links are formed through one-to-one bargaining and negotiation. For any node, the benefit of cooperation comes not just from nodes directly connected (one-hop), but also from nodes that are indirectly connected (multi-hop, through other users). For instance, in multi-hop wireless networks, this is the incentive the users have for forwarding packets. In other words, by activating a communication link towards one of their neighbors, they gain by having access to the users with which that neighbor has activated his links, and so on, recursively. The more users a user has access to, the more desirable it is for his neighbors to activate their link towards him. Therefore, in this paper, we study cooperation and games based on the notion of coalitions. The concept of users being connected to each other, and – by getting connected – acquiring access to all the other users that each of them had so far access to, can be well captured by cooperative game theory (also known as coalitional game theory [3]). A question that has only relatively recently began to attract attention ([4] is the first work in this area) is the actual way the coalition is formed. There has been extensive research on coalition formation in the context of social and economic networks [5], [6]. The cooperative game is usually modelled as a two-period structure. Players must first decide whether or not to join a coalition. This is done by pairwise bargaining, in which both players have to agree to join in a coalition. In the second step, players in the coalition negotiate the payoff allocation. The central problem is to study the payoff allocation scheme and whether the scheme results in a stable solution. In our previous work [7], we studied such two- phase games in the context of communication networks and investigated the fundamental tradeoffs between the gain and cost of collaboration. In this paper, we study another type of iterated games, where the payoff of players depends on the coalition structure they belong to, and where the payoff changes with the procedure of coalition formation. A learning strategy is introduced to guarantee that the game converges to the Nash equilibrium. We also investigate the condition for the core of the coalitional game being nonempty. The rest of the paper is organized as follows: Section II describes the mathematical framework within which we deal with the concepts just discussed. The terminology we use in the paper is defined. Section III presents various gain and cost models that can be used in communication networks. In Section IV-B we present the learning strategy that drives 978-1-4244-4177-8/09/$25.00 ©2009 IEEE 10 Authorized licensed use limited to: University of Maryland College Park. Downloaded on August 5, 2009 at 16:18 from IEEE Xplore. Restrictions apply.