CONCURRENCY AND COMPUTATION: PRACTICE AND EXPERIENCE Concurrency Computat.: Pract. Exper. 2007; 19:983–998 Published online 28 September 2006 in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/cpe.1090 Management of real-time streaming data Grid services Geoffrey Fox, Galip Aydin, Hasan Bulut, Harshawardhan Gadgil, Shrideep Pallickara, Marlon Pierce ∗,† and Wenjun Wu Community Grids Laboratory, Indiana University, 501 North Morton Street, Suite 224, Bloomington, IN 47404, U.S.A. SUMMARY We discuss our message-based approach to managing real-time data streams and building higher level services to produce and consume them. Our messaging system acts as a substrate that can be used to provide qualities of service to various streaming applications ranging from audio–video collaboration systems to sensor Grids. The messaging substrates are composed of distributed, hierarchically arranged message broker networks. Services such as filters are deployed along the edges of the network. We discuss the role of management systems for both broker networks and filter services: broker network topologies must be created and maintained, and distributed filters must be arranged in appropriate sequences. These managed broker networks may be applied to a wide range of problems. We discuss applications to audio–video collaboration in some detail and also describe applications to streaming Global Positioning System data streams. These provide specific application filters that can transform and republish message streams to the broker system. Copyright c  2006 John Wiley & Sons, Ltd. Received 13 March 2006; Accepted 4 May 2006 KEY WORDS: real-time systems; Web services; Grid computing 1. INTRODUCTION A growing number of applications involve real-time streams of information that need to be transported in a dynamic, high-performance, reliable, and secure fashion. Examples include sensor nets for both science and military applications, mobile devices on ad-hoc networks, and collaborative applications. ∗ Correspondence to: Marlon Pierce, Community Grids Laboratory, Indiana University, 501 North Morton Street, Suite 224, Bloomington, IN 47404, U.S.A. † E-mail: marpierc@indiana.edu Contract/grant sponsor: U.S. Department of Energy (SciDAC Program); contract/grant number: DE-FC02-02ER2615 Contract/grant sponsor: NASA (Advanced Information Systems Technology Program); contract/grant number: 1253656 Copyright c  2006 John Wiley & Sons, Ltd.