446 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 5, NO. 2, FEBRUARY 2006 Optimal and Suboptimal Packet Scheduling over Correlated Time Varying Flat Fading Channels Ashok K. Karmokar, Student Member, IEEE, Dejan V. Djonin, and Vijay K. Bhargava, Fellow, IEEE Abstract— We address the issue of optimal packet scheduling over correlated fading channels which trades off between min- imization of three goals: average transmission power, average delay and average packet dropping probability. We show that the problem forms a weakly communicating Markov decision process and formulate the problem as both unconstrained and constrained problem. Relative value iteration (RVI) algorithm is used to find optimal deterministic policy for unconstrained prob- lem, while optimal randomized policy for constrained problem is obtained using linear programming (LP) technique. Whereas with RVI only a finite number of scheduling policies can be obtained over the feasible delay region, LP can produce policies for all feasible delays with a fixed dropping probability and is computationally faster than the RVI. We show the structure of optimal deterministic policy in terms of the channel and buffer state and form a simple log functional suboptimal scheduler that approximately follows the optimal structure. Performance results are given for both constant and bursty Poisson arrivals, and the proposed suboptimal scheduler is compared with the optimal and channel threshold scheduler. Our suboptimal scheduler performs close to the optimal scheduler for every feasible delay and is robust to different channel parameters, number of actions and incoming traffic distributions. Index Terms— Packet scheduling, Markov decision process, wireless correlated fading channel, suboptimal scheduling. I. I NTRODUCTION M ODERN and future wireless networks will support numerous services with a wide range of quality of service (QoS), such as delay, rate, bit error rate (BER), packet dropping etc., requirements. Time-varying nature of the radio channel poses a challenging task of delivering such a wide variety of services. Several adaptive techniques have been developed in practice and in literature to compensate channel’s time-variation. These adaptive techniques include changing transmission power levels, modulation constellations, coding rates, or a combination of these mentioned parameters [1]. The quality of the wireless channel at any instant depends on the channel state at previous instant due to significant degree of correlation of the fading process [2]. In wireless networks, mobile devices usually rely on a battery with a limited amount of energy. So, minimization of transmission power can lead to Manuscript received January 26, 2004; revised November 22, 2004 and April 6, 2005; accepted May 1, 2005. The associate editor coordinating the review of this paper and approving it for publication was Zhensheng Zhang. This work is supported by the Natural Sciences and Engineering Research Council (NSERC) of Canada under a strategic project grant. The material in this paper was presented in part at the IEEE International Conference on Communications (ICC 2004), June 20-24, 2004, Paris, France. The authors are with the Department of Electrical and Computer Engi- neering, University of British Columbia, Vancouver, BC, V6T 1Z4, Canada (e-mail: ashokk@ece.ubc.ca; ddjonin@ece.ubc.ca; vijayb@ece.ubc.ca). Digital Object Identifier 10.1109/TWC.2006.02024. Fig. 1. Schematics of the system and channel model. more efficient utilization of battery energy and hence longer battery life of mobile devices. We consider the situation depicted in Fig. 1 (a) where packets arrive from some higher layer application and are placed into a finite transmission buffer. Periodically, a scheduler takes some of the packets from the buffer, encodes them, and transmits the encoded packets over a correlated fading channel. We assume that current channel state information (CSI) and buffer state are available at the transmitter. By delaying packet in buffer, the scheduler can save power. However, different users can have different QoS (i.e., delay, packet dropping, etc.) requirements, and allowing excessive delays can result in buffer overflows and hence packet dropping. In this paper, we study the trade- off between average transmission power, delay and packet dropping probability for different transmission schemes over a fading channel with memory. Traditionally, two sets of issues have been analyzed in virtual isolation from each other in the literature: some authors focus mainly on network-layer throughput, delay and packet dropping, while others concentrate on physical-layer channel modeling, coding, modulation and detection. Recently, it has been realized that the system performance can be improved significantly by adapting transmission parameters with both layer parameters simultaneously. Several research papers that have addressed cross layer issues are [3]- [12]. Early work of [3] gives a dynamic-programming framework for trans- mission policies over a two-state Gilbert-Elliott channel with constraint on average delay and peak power. An offline and online packet scheduling schemes for an additive white Gaussian noise channel have been analyzed in [4] with the goal of energy minimization subject to deadline and average delay constraint respectively. A similar type of problem based on dynamic programming is addressed in [5], where data throughput is maximized with fixed amount of energy and 1536-1276/06$20.00 c  2006 IEEE