EXTENSION OF SMIL WITH QOS CONTROL AND ITS IMPLEMENTATION Yoshiki Terashima Keiichi Yasumoto Teruo Higashino Kota Abe Toshio Matsuura Kenichi Taniguchi Graduate School of Engineering Science, Osaka Univ. Toyonaka, Osaka 560-8531, JAPAN Dept. of Information Processing and Management, Shiga Univ. Hikone, Shiga 522-8522, JAPAN Media Center, Osaka City Univ. Sumiyoshi-ku, Osaka 558-8585, JAPAN ABSTRACT In this paper, we propose a language called QOS-SMIL where some QoS guarantee statements are added to a subclass of SMIL, and give its implementation technique. In QOS-SMIL, in addition to standard facilities of SMIL, (1) dynamic media-scaling of each multi-media object, and (2) explicit inter-media synchronization among objects, can be specified. In the proposed implementation technique, we use real-time LOTOS as an intermediate language so that we convert a given QOS-SMIL description to the corre- sponding real-time LOTOS specification, and implement it with our real-time LOTOS compiler. Through some experiments, we have confirmed that our approach has some advantages w.r.t. de- velopment cost of multimedia-systems with QoS control. 1. INTRODUCTION In order to play back multi-media objects (e.g., video, audio, text, etc) effectively to users, there has been great demand for multime- dia description languages which can provide various way of pre- sentation with multiple multimedia objects. For such purposes, W3C has standardized SMIL 1.0 [7] which has facilities to specify display location and playback timing of objects and to control tem- poral ordering of multiple object playbacks. Unfortunately, SMIL 1.0 has no facility to specify QoS guarantees (e.g., media-scaling of objects in case of resource shortage, keeping maximum skew deviation in inter-media synchronization within a threshold, etc). Therefore, it may be difficult to apply SMIL 1.0 directly to the de- velopment of distributed multimedia systems where available net- work/system resources are dynamically changing. In this paper, we propose QOS-SMIL which has some QoS guarantee statements in addition to the basic facilities of SMIL 1.0. The new features of QOS-SMIL are (1) a dynamic switching facility to alternative objects, and (2) a precise inter-media syn- chronization facility for keeping maximum skew deviation within a specified threshold, and so on. The above facility (1) enables media-scaling of objects (i.e., based on the priority and system load, the object with the lowest priority is dynamically switched to the one with lower quality) and dynamic server selection (i.e., if object locations are specified by URLs, the reception of object data is dynamically switched to the URL with no congestion when network congestion occurred). For QOS-SMIL implementation, we need to implement QoS control and media synchronization mechanisms in addition to con- trol mechanisms of object playbacks and temporal ordering among object playbacks. Thus, we propose an implementation technique with real-time LOTOS [4, 5] which has some new facilities such as specifying timing constraints to each event, in addition to pow- erful control operators (e.g., parallel, synchronization, sequential, choice, interruption among multiple processes) of ISO standard specification language LOTOS [2]. With real-time LOTOS, a sys- tem can be composed of a main process and some constraint pro- cesses where the main process just specifies events and their tem- poral ordering, and the constraint processes specify constraints for the events in the main process: for example, execution timing of some events to meet the intended QoS guarantee. We have developed a converter from QOS-SMIL to real-time LOTOS. Using this converter and our real-time LOTOS compiler [4, 5], we have generated programs from QOS-SMIL documents and evaluated efficiency. As a result, we have confirmed that the generated programs could achieve at least 80% of execution ef- ficiency and about half of development cost (w.r.t. description amount) compared with programs coded by C and a multi-thread mechanism. We could also achieve the maximum skew in inter- media synchronization within about 20 ms under normal UNIX. 2. SMIL AND ITS EXTENSION 2.1. Outline of SMIL1.0 and target sub-class In a SMIL document, we can specify both layout (e.g., display, position, etc of each object) and timing (e.g., starting and ending time, duration, etc) for each time-dependent media object (e.g., video, audio, etc) and time-independent media object (e.g., picture images, text, etc). Hereafter, we call both types of objects simply as objects. Similarly to XML and HTML, a SMIL document is composed of elements denoted by <element> ... </el- ement>. We can also give some attributes to each element by <element attr name=attr value>. SMIL has some con- trol mechanisms denoted by par, seq and repeat elements, to play back multiple objects in parallel, sequentially and iteratively (nesting of those elements is also allowed). SMIL also has the switch element which allows static selection among alternative objects depending on system-specific information (e.g., maximum available bandwidth, display size, etc). In this paper, we treat the following elements and attributes as a subclass of SMIL 1.0. elements attributes region id,left,top,width,height,title video,audio,img,text id,begin,end,dur,clip-begin, clip-end,src,region,fill,repeat par id,begin,end,dur,endsync,repeat seq id,begin,end,dur,repeat Here, location of each object data is specified by its file name or URL (e.g., http://somehost.somedomain.com/video1.mpg). For the sake of simplicity, we do not treat layout-specific elements such as root-layout, fit, ref nor the alt element which specifies alternative text. 2.2. QOS-SMIL In QOS-SMIL, in addition to the above subclass of SMIL 1.0, we can use (1) the dswitch element for dynamic switching to alter- native objects, (2) the pri attribute for specifying priority to each object, and (3) the maxskew attribute for precise inter-media syn- chronization. (1) dynamic switching to alternative objects The dswitch element enables dynamic selection and playback among speci- fied alternative objects depending on variation of available sys- tem/network resources. Major difference from switch element