Quality-Driven TCP Friendly Rate Control for Real-Time Video Streaming Haiyan Luo, Dalei Wu, Song Ci Department of Computer and Electronics Engineering University of Nebraska-Lincoln Email: {hluo, dwu}@bigred.unl.edu, sci2@unl.edu Antonios Argyriou Philips Research Eindhoven, The Netherlands Email: antonios.argyriou@philips.com Haohong Wang Marvell Semiconductors Santa Clara, CA 95054 Email: haohongw@marvell.com Abstract—TCP Friendly Rate Control (TFRC) has been de- signed to provide smoother sending rate than TCP for multimedia applications. However, most existing work on TFRC is restricted within exploring the performance of TFRC itself in wired or wireless networks without considering the interaction between TFRC and other network layers. This paper proposes a quality- driven TFRC framework for real-time video streaming, where real-time video coding at the application layer and the packet sending rate at the transport layer are jointly optimized. The proposed framework is formulated to find the optimal video coding parameters and the sending rate to minimize the end- to-end expected video distortion under a given video playback delay constraint. The proposed framework has been implemented and tested by using both H.264/AVC codec and NS-2 simulator. Experimental results demonstrate that the proposed joint opti- mization framework can significantly improve the received video quality over the existing schemes, especially when delay bound is tight. I. I NTRODUCTION As the current dominant transport protocols, both UDP and TCP have their drawbacks for video applications. UDP, though simple and efficient, lacks the mechanism to guarantee delivery. Thus, it is up to the receiving application to detect loss or corruption and recover data using error correction techniques. If data is lost, the stream may suffer a dropout. Furthermore, UDP streams are often more difficult to penetrate firewalls. On the other hand, TCP, whose transmission rate is controlled by a congestion window which is halved for every window of data containing a packet drop and increased by roughly one packet per window of data otherwise, is not well-suited for real-time streaming applications, due to its rate variability and additional delays introduced by reliability mechanisms, although it is efficient for bulk data transfer. Nevertheless, congestion control, typically in the form of rate- based congestion control for streaming applications, is still of primary importance for multimedia applications to deal with the diverse and constantly changing conditions of the Internet [1]. In essence, the goal of congestion control algorithms is to prevent applications from either overloading or under-utilizing the available network resources. TCP-Friendly protocols emerge as a good tradeoff between TCP and UDP, which not only boast slow responsiveness in order to smooth data throughput, but also are TCP-friendly. The definition of ”TCP-friendliness” is that a traffic flow, in steady state, uses no more bandwidth in the long term than a conforming TCP flow would use under comparable conditions [2], thus avoiding the starvation or even congestion collapse to TCP traffic when the two types of traffic coexist. In other words, it is characterized by the same behavior with TCP in the long run, but allows smoother throughput fluctuations. As one of the most popular TCP-Friendly protocols, TFRC is an equation-based congestion control mechanism that uses the TCP throughput function derived in [1] to calculate the actual available rate for a media stream. Since it was formally introduced in [3] and standardized in [8], there has a lot of research [2]–[7] done on TFRC, due to its well-known advantages and bright prospects. However, most of these research activities only focus on the performance of TFRC per se or the comparison with its transport layer peer - TCP. Some work on video streaming using TFRC only focuses on the TFRC performance with pre-encoded video streams as stated in [9], [10]. This paper is the first work to jointly integrate TFRC with real-time video encoding in a cross-layer optimization approach. Our solution aims at setting up a bidirectional interaction between source online encoding and network TFRC. For example, if there is a heavy traffic in the network such that rate control cannot be effectively performed. This information can be used by the video encoder to adapt its coding parameters and decrease the coding rate to alleviate the network congestion, while maintaining a possibly best received video quality at the receiver side. Similarly, in a video sequence to be transmitted, some video clips or video frames with large content varying due to high video mobility might have to be represented by a large amount of data after compressed. The rate control at the transport layer can use this information to increase its sending rate to avoid a large packet loss rate due to possible queue overflow or network congestion. The proposed framework is formulated as to select the optimal video coding parameters and the transport layer sending rate to minimize the average video distortion with a predetermined delay bound. To provide a smooth video playback experience, the video delay bound refers to the video frame delay deadline. Different video frames are associated This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE "GLOBECOM" 2008 proceedings. 978-1-4244-2324-8/08/$25.00 © 2008 IEEE.