A Scalable Peer-to-Peer Architecture for Distributed Information Monitoring Applications Bugra Gedik, Student Member, IEEE, and Ling Liu, Member, IEEE Computer Society Abstract—We present PeerCQ, a decentralized architecture for Internet scale information monitoring using a network of heterogeneous peer nodes. PeerCQ uses Continual Queries (CQs) as its primitives to express information-monitoring requests. The PeerCQ development has three unique characteristics. First, we develop a systematic and serverless approach to large scale information monitoring, aiming at providing a fully distributed, highly scalable, and self-configurable architecture for scalable and reliable processing of a large number of CQs over a network of loosely coupled, heterogeneous, and possibly unreliable nodes (peers). Second, we introduce an effective service partitioning scheme at the P2P protocol layer to distribute the processing of CQs over a peer-to-peer information monitoring overlay network while maintaining a good balance between system utilization and load balance in the presence of peer joins, departures, and failures. A unique feature of our service partitioning scheme is its ability to incorporate strategies for handling hot spot monitoring requests and peer heterogeneity into the load balancing scheme in PeerCQ. Third, but not least, we develop a dynamic passive replication scheme to enable reliable processing of long-running information monitoring requests in an environment of inherently unreliable peers, including an analytical model to discuss its fault tolerance properties. We report a set of experiments demonstrating the feasibility and the effectiveness of the PeerCQ approach to large-scale peer-to-peer information monitoring. Index Terms—Distributed information monitoring, peer-to-peer networks, continual query systems. æ 1 INTRODUCTION P EER-TO-PEER (P2P) systems are massively distributed computing systems in which peers (nodes) commu- nicate directly with one another to distribute tasks, exchange information, or share resources. There are currently several P2P systems in operation and many more are under development. Gnutella [6] and Kazaa [8] are among the most prominent first generation peer-to-peer file sharing systems operational today. These systems are often referred to as unstructured P2P networks and they share two unique characteristics. First, the topology of the overlay network and the placement of the files within the network are largely unconstrained. Second, they use a decentralized file lookup scheme. Requests for files are flooded with a certain scope. There is no guarantee of finding an existing file within a bounded number of hops. The random topology combined with flooding-based routing is clearly not scalable since the load on each peer grows linearly with the total number of queries in the network, which in turn grows with the size of the system. Chord [19], Pastry [16], Tapestry [20], CAN [14] are examples of the second generation of peer-to-peer systems. Their routing and location schemes are structured based on distributed hash tables. In contrast to the first generation P2P systems such as Gnutella and Kazaa, these systems provide guaranteed content location (persistence and availability) through tighter control of the data placement and the topology construction within a P2P network. Queries on existing objects are guaranteed to be answered in a bounded number of network hops. Their P2P routing and location schemes are also considered more scalable. These systems differ from one another in terms of their concrete P2P protocol design, including the distributed hash algorithms, the lookup costs, the level of support for network locality, and the size and dependency of routing table with respect to the size of the P2P overlay network. Surprisingly, many existing P2P protocols [1], [19], [14], [16], [6] do not distinguish peer heterogeneity in terms of computing and communication capacity. As a result, these protocols distribute tasks and place data to peers assuming all peers participate and contribute equally to the system. Work done in analyzing characteristics of Gnutella in [18] shows that peers participating in these systems are heterogeneous with respect to many characteristics, such as connection speeds, CPU, shared disk space, and peers’ willingness to participate. These evidences show that P2P applications should respect the peer heterogeneity and user (application) characteristics in order to be more robust [18]. In this paper, we describe PeerCQ [5], a peer-to-peer information monitoring system which utilizes a large set of heterogeneous peers to form a peer-to-peer information monitoring network. Many application systems today have the need to track changes in multiple information sources on the Web and notify users of changes if some condition over the information sources is met. A typical example in the business world is to monitor availability and price information of specific products, such as “monitor the price of 5 mega pixel digital cameras during the next two months and notify me when one with price less than $500 becomes available,” “monitor the IBM stock price and notify me when it increases by 5 percent.” In a large scale information monitoring system [9], many users may issue the same information monitoring request, such as tracking IBM stock price changes during a given period of time. We call such IEEE TRANSACTIONS ON COMPUTERS, VOL. 54, NO. 6, JUNE 2005 767 . The authors are with the College of Computing, Georgia Institute of Technology, Atlanta, GA 30332. E-mail: {bgedik, lingliu}@cc.gatech.edu. Manuscript received 21 Oct. 2003; revised 23 Nov. 2004; accepted 30 Nov. 2004; published online 15 Apr. 2005. For information on obtaining reprints of this article, please send e-mail to: tc@computer.org, and reference IEEECS Log Number TC-0187-1003. 0018-9340/05/$20.00 ß 2005 IEEE Published by the IEEE Computer Society