978-1-4244-7493-6/10/$26.00 ©2010 IEEE ICME 2010 A REMOTE THIN CLIENT SYSTEM FOR REAL TIME MULTIMEDIA STREAMING OVER VNC Kheng-Joo Tan 1 , Jia-Wei Gong 1 , Bing-Tsung Wu 1 , Dou-Cheng Chang 1 , Hsin-Yi Li 1 , Yi-Mao Hsiao 2 , Yung-Chung Chen 1 , Shi-Wu Lo 1 , Yuan-Sun Chu 3 and Jiun-In Guo 1 1 Department of Computer Science, National Chung Cheng University, Taiwan, R.O.C. 2,3 Department of Electrical Engineering, National Chung Cheng University, Taiwan, R.O.C. 1 {ccj96m, kcw97m, wupt, ctc97m, lyh97m, u93410014, shiwulo, jiguo}@cs.ccu.edu.tw , 2 93mowmow@vlsi.ee.ccu.edu.tw , 3 chu@ee.ccu.edu.tw ABSTRACT This paper proposes a remote thin client system for real time multimedia streaming over VNC. A remote frame can be split as two parts, i.e. high motion part and low motion part, and transmitted through the Internet from servers to clients according to the proposed hybrid RTP protocol. A Dynamic Image Detection Scheme (DIDS) is proposed to automatically detecting the high motion part of a frame with only 1% of extra CPU loading. In addition, an Error Detection Scheme (EDS) and a Dynamic Bit-rate Control Scheme (DBCS) are also proposed to ensure good video streaming quality under bandwidth limited applications. This paper also proposes a linear time BU-level rate control algorithm to ensure the proposed DBCS can be finished in real time. The proposed algorithm reduces the computational complexity from ) ( 2 n O to be ) (n O . By using the proposed thin client system, we can achieve about 22 fps of real-time SIF video streaming with good video quality under 32 KByte/s of bandwidth limitation, which speeds up about 172 times in remote frame display when compared to pure VNC. Keywords— Thin client, VNC, Image detection, Video streaming. 1. INTRODUCTION In thin client systems, clients are mostly light weight portable devices with low computing power. They are mainly responsible for displaying graphical output and handling user events such as keyboard or mouse events. In contrast to clients, servers are powerful to handle more complicated operations such as image capturing, image encoding, and so on. Remote desktop is a kind of thin client systems using Remote Frame Buffer Protocol (RFB) or Remote Desktop Protocol (RDP) to remotely control servers by clients like personal computers, notebooks, PDAs, and even smart phones. This mechanism allows thin clients to keep light weight while sharing the computing power with servers. But in most applications especially in displaying multimedia contents, real time transmission is the main design challenge caused by extremely high data bandwidth. Thus, the current thin client systems adopting VNC cannot achieve a smooth display on multimedia contents remotely through thin clients under the limited data bandwidth. In order to solve this problem, there have been some solutions proposed in the literature [1-3]. In [1], high/low motion scenarios are tested on remote thin client systems. The experimental results show that in high motion scenario such as video streaming and 3D gaming, extra hardware is a must on both the server and client sides to resolve the high bandwidth problem. In [2] and [3], similar hybrid remote thin client system structures with different motion detection algorithms are used to solve the real time streaming problem mentioned in [1]. In this kind of structure, remote frames are classified into high and low motion modes. A pure software mode decision algorithm is proposed for deciding frame motion mode in [2]. As for [3], an extra decision abstraction layer is added in the device driver to decide whether the remote frame is in high or low motion mode. As shown in Fig. 1, if the remote frame is belonging to high motion mode, the H.264 encoder is used to encode the frame before sending it to clients. On the contrary, if the remote frame is belonging to low motion mode, the traditional RFB protocol is used to update frame buffer pixels of the client devices. In other words, either high or low motion mode will be chosen before a remote frame is sent to clients. Fig. 1: The hybrid thin client system in [3] 992 978-1-4244-7492-9/10/$26.00 ©2010 IEEE ICME 2010