DirectShow RTP Support for Adaptivity in Networked Multimedia Applications Linda S. Cline, John Du, Bernie Keany, K. Lakshman, Christian Maciocco, David M. Putzolu 1 Intel Architecture Labs 2 2111 N.E 25 th Avenue Hillsboro OR, 97124 1 Author names are in alphabetical order. 2 linda.cline@intel.com, john.du@intel.com, bernie.keany@intel.com, lakshman@jf.intel.com, christian.maciocco@intel.com, David.Putzolu@intel.com Abstract Execution of a collaboration application or a distributed game involves many individual multimedia applications that concurrently generate and consume audio and video streams. As the resource requirements of individual streams change and as streams/applications are started or terminated, the amount of available resources may change dynamically. Networked MultiMedia applications must be willing to adapt to these changes by taking advantage of the fact that they can deliver varying levels of service that are acceptable to users. This paper addresses the problem of adding network and host adaptive capability to DirectShow RTP. DirectShow is Microsoft’s architecture for capture and presentation of multimedia data. DirectShow RTP is a framework that extends the DirectShow architecture, adding support for streaming multimedia data using the RTP protocol. We have extended this framework by adding support for streaming applications which dynamically compensate for varying resource availability on the local host and on the computer networks being used to deliver multimedia data to and from these applications. The lessons learned are useful both to applications and to designers of frameworks used to build such applications. 1. Introduction Networked MultiMedia (NetMM) applications are one of the most resource intensive classes of software executed on personal computers and workstations. Such applications have high requirements in terms of host processing power and network bandwidth. These applications also often require that the resources necessary for execution be provided by the underlying operating system and computer networks within near-real time constraints. Because both resource requirements and availability change dynamically, NetMM applications must be prepared to adapt to these changes in a graceful fashion [11,12]. In this paper, we focus on two types of adaptation - network adaptation and host adaptation. Network adaptation is the ability of an application to vary the way it utilizes network resources so as to make most efficient use of the network in the face of such conditions as changing loss, jitter, and available bandwidth. Host adaptation is defined as the ability of an application to modify its behavior based on conditions present in the local host, including such factors as CPU utilization and available memory. The following examples are illustrative of situations where both types of adaptation are useful: Presentation viewing by clients with heterogeneous network and processor resources. In a video conference or collaboration session that involves a set of users with heterogeneous capabilities in terms of processor power and network bandwidth availability, it may not be possible for all endpoints to receive all streams. In this situation, all members of a session are able to participate if multiple streams of hierarchically encoded video are used. This form of adaptation allows significantly greater heterogeneity of participants in comparison to a presentation consisting of a single video stream. Competition for resources among multiple streams. Consider an application that uses an audio stream, a high-bitrate video stream, and a bursty slide-show stream, where the audio stream is